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(57) Abstract 

A smart tracking system includes a digital processor for se- 
lectively processing error signals representing the angular error 
between a tracked subject and the orientation of the tracking de- 
vice/camera. In one embodiment, the user selects a set of data va- 
lues representing the operating parameters of the system in a spe- 
cific operational mode. The processor uses the selected variables 
to perform a series of computations which selectively modify the 
angular error signals in a manner that will produce a video pic- 
ture which is visually and aesthetically pleasing to human viewers. 
In a second embodiment, the set of data values characterizing the 
operational mode of the system automatically are selected by the 
processor based on the past history of the subject's motion and 
the current operation mode of the system. 
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SMART TRACKING SYSTEM 

Related Applications 

This application is co-pending with United 
States Patent Application Serial No. 07/396,987, 
filed August 22, 1989, now United States patent 
4,980,871, entitled Ultrasonic Tracking System, the 
subject matter of which is incorporated herein by 
reference. 

Field of the Invention 

The present invention relates to measuring and 
tracking systems and, more particularly, to a system 
wherein the manner in which a camera tracks a moving 
subject is dependent upon the context and history of 
the subject's movements. 

Background of the Invention 

When an automatic tracking system, such as that 
disclosed in U.S. Patent 4,980,871, directs a video 
camera for human viewers, there are a number of 
aesthetic factors which should be considered. In 
particular, factors such as the centering of the 
subject within the camera frame, the smoothness of 
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the camera motion against a stable background, and 
the continuity of the direction of camera motion are 
all elements which make a video picture 
aesthetically pleasing to human viewers. 
Conversely, a video picture which is characterized 
by abrupt or continuous camera motion and 
inconsistent camera tracking speed has a tendency to 
violate a viewer's visual satisfaction criteria. 

A typical automated tracking system utilizes a 
two step process, to eliminate any errors between the 
position of a tracked subject and the camera 
direction. First, the tracking system measures the 
angular error between the tracked subject and the 
orientation of the tracking device/camera. Next, 
the error information is used to generate a 
correction signal to control the motion of the 
tracking device/camera. The new camera position is 
then utilized in calculating future angular errors. 
The resulting tracking is mechanical with a tendency 
to be just as annoying to the viewer as if the 
tracking was very inaccurate. For example, constant 
readjustment of the camera position to every motion 
of the subject can cause rapid eye fatigue in the 
viewer . 

Additionally, typical automated tracking systems 
use the same tracking strategy and style for all 
subjects and contexts, whether the subject is giving 
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a lecture or participating in a fast-paced athletic 
event. The inability of such tracking systems to be 
set to, or alternate between, different styles 
severely limits their applications. 

Unlike an automated tracking system, a human 
cameraman directs a camera to a subject with less 
than complete accuracy. The cameraman further 
adjusts the tracking style according to the context 
in which the subject moves and the immediate past 
history of the subject's motions. The resulting 
video contains minor tracking inaccuracies which are 
visually more pleasing to a human viewer than the 
exacting and mechanical tracking of an automated 
system. 

Accordingly, it is an object of the present 
invention to provide a system which allows a camera 
to track a moving subject in a manner which is 
visually and aesthetically pleasing to human 
viewers, by selectively permitting angular errors 
between the position of the subject and the 
direction in which the tracking device/camera is 
pointing . 

Another object of the present invention is to 
provide a system which allows a camera to track a 
moving subject in a specific style which is 
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appropriate to the context in which the subject is 
moving . 

A further object of the present invention is to 
provide a system which allows a camera to track a 
moving subject in a style which is determined by the 
system, based on the past history of the subject's 
motions . 

Summary of the Invention 

A system for permitting a device to point at and 
track the movements of a subject comprises a signal 
source and a control means which receives signals 
from the source and generates therefrom control 
signals. A memory means stores plural sets of data 
values, each of the sets representing the operating 
parameters of said system in a specific operational 
mode. Weans are provided for selecting one of said 
plural sets of data values. A processor processes 
the control signals in accordance with the selected 
data values and produces processed control signals. 
A drive unit moves the device about an axis in 
response to the processed control signals to 
maintain the device pointed at the subject. 

In one embodiment/ the means for selecting one 
of the plural sets of data values comprises a 
user-defined signal for indicating the desired data 
value set and its corresponding operational 
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parameters. In a second embodiment, the means for 
selecting a data value set is a function of the 
processor which, when processing the control 
signals, determines whether the selected data value 
set is appropriate for the moving subject. If not, 
the processor selects a new data value set and 
thereby changes the operational mode of the system. 

The forgoing and other features, objects and 
advantages of the invention will be more fully 
understood by reading the detailed description 
below, in conjunction with, the drawing. 

Brief Description of the Drawing 

In the drawing: 

FIG. 1 is a pictorial illustration of the 
drive unit of a tracking system suitable for use 
with the present invention; 

FIG. 2 is a diagrammatic illustration of 
the subject, such as an athlete, wearing the 
transmitter unit of a tracking system suitable 
for use with the present invention; 

FIG. 3 is a functional block diagram of a 
drive unit according to the invention; 

FIG. 4 is a flow chart of the error 
correction process of prior art automatic 
tracking systems; 

FIG. 5 is a flow chart of the error 
correction process executed by the drive unit of 
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FIG. 3 according to a first embodiment of the 
pr es ent invent i on ; 

FIG. 6A is a flow chart of the algorithmic 
steps for calculating the dead zone of the 
process of FIG. 5; 

FIG. 6B is a flow chart of the algorithmic 
steps for shifting the dead zone of the process 
of FIG. 5; 

FIG. 6C is a flow chart of the algorithmic 
steps for continuity control of the process of 
FIG. 5; 

FIG. 6D is a flow chart of the algorithmic 
steps for motion control of the process of FIG. 
5; 

FIG. 7 is a flow chart of the error 
correction process executed by the drive unit of 
FIG. 3 according to a second embodiment of the 
present invention; and 

FIG. 8 is a flow chart of the algorithmic 
steps for automatic context determination of the 
process of FIG. 7. 

Detailed Description 

The present invention provides a smart tracking 
system which in an exemplary embodiment, is an 
active tracking system comprising two units. The 
first unit is a small, battery-operated, 
omni-directional ultrasonic transmitter which is 
carried by the subject to be tracked. The 
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transmitter produces brief, bursts of sound at 
frequencies above the range of human hearing. The 
second unit comprises a motorized camera drive 
capable of tilting and panning an attached camera. 
It should be understood that the present invention 
may also be implemented with a passive tracking 
system, not requiring a transmitter worn by the 
subject . 

In an exemplary embodiment, a battery-operated 
ultrasonic receiver is attached to the camera 
drive. The receiver is tuned to the same frequency 
as the transmitter carried by the subject. The 
receiver employs three ultrasonic microphones, 
together with electronic circuitry to detect and 
amplify the ultrasonic- signals generated by the 
transmitter and intercepted by the microphones. The 
microphones are located at three of the four corners 
of an imaginary square centered on and perpendicular 
to the axis of the camera lens. The signals 
emanating from the subject will arrive at all three 
microphones simultaneously only if the camera is 
pointed directly at the subject. The receiver 
measures the relative arrival times of signals 
received by the microphone, determines the direction 
and degree to which the camera is not pointed 
directly at the subject, and causes a servo-system 
to drive motors to re-orient the camera (and 
receiver) to point toward the subject matter. This 
re-orientation or tracking is done in a manner that 
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is visually pleasing to human viewers, and is 
adaptable to a variety of contexts. 

Referring to FIG. 1, a television or 
motion-picture camera or other device 10 (hereinafter 
referred to as a camera, without loss of generality) 
is attached to a platform or mount which is part of 
a motorized drive unit, generally designated 12, 
that is in turn mounted to a tripod 14 or other 
support. Drive unit 12 can tilt camera 10 up and 
down about horizontal axis 16 from about 30° below 
the horizontal to about 30° above the horizontal. 
The drive unit can also pan the camera left and 
right about vertical axis 18 through repeated 36 0° 
circles. The drive unit responds to ultrasonic 
signals received via microphones 20A, 20B and 20C to 
control direct-current motors (not shown) to move 
the camera mount . 

A subject 22 as shown in FIG. 2, who is to be 
recorded (e.g., an athlete), is provided with an 
ultrasonic transmitter 24 to be worn or carried on 
his or her person. The transmitter unit may 
include, for example, an elastic belt 26 with buckle 
for wearing around the waist of the subject. 
Attached to the belt are three or more transducer 
units (only one transducer unit 28, being shown in 
the drawing) connected by flexible cables embedded 
in belt 26. 
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Drive unit 12 includes two direct-current motors 
42 and 44, respectively. Through conventional 
mechanisms, motor 42 tilts the camera mount up and 
down and the motor 44 pans the camera mount left and 
right. Since the camera is securely fastened to the 
mount, it undergoes the same motions. The motors 
are driven by electronic circuitry located on the 
circuit boards (not shown in FIG. 2) inside the 
drive unit housing. The circuitry is powered by 
batteries (also not shown). 

Three detector circuits 30A, 30B and 30C are 
shown in the block diagram of FIG. 3, one for each 
of the three ultrasonic transducers 20A, 20B and 
20C, respectively. Also shown are two motor driver 
circuits 32 and 34, one for each of the two DC 
motors 42 and 44, respectively. A controller 
circuit 36 and digital processor 46 operatively 
connect the detectors to the motor drivers. Each of 
the transducers 20A, 20B and 20C receive the 
ultrasonic bursts emitted by the transmitter unit 
and supplies a corresponding electrical signal to 
its associated detector circuit. The detector 
circuit detects the presence of an ultrasonic burst 
and produces a rectangular pulse whose leading edge 
coincides with the beginning of the detected burst. 
These pulses, 50A, 50B and 50C are routed to 
controller 36, which measures the delay between the 
pulses from the three detectors and loads digital 
error registers (not shown) with binary signals, 
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VERB, and HERR. These 4-bit signals represent the 
angular error along the vertical and horizontal axes, 
respectively, between the camera direction and the 
subject. VERR and HERR are updated each, time an 
ultrasonic burst is received. 

The structure and function of transmitter unit 
24, and drive unit 12, including direct-current 
motors 42 and 44, and the manner in which the drive 
unit 12 tracks the location of transmitter unit 24 
to keep the camera pointed at the subject, is 
discussed in detail in U.S. Patent 4,980,871, which 
is incorporated herein by reference, and will not be 
repeated for brevity. 

Digital processor 46 receives the VERR and HERR 
signals over busses 52 and 54, respectively. 
Processor 46, may be a microprocessor. One such 
microprocessor suitable for use with the present 
invention is the model MC 68705C8 available from 
Motorola, Inc., Phoenix, Arizona. Processor 46 
manipulates the values of VERR and HERR, in a manner 
to be described more fully below, according to ttie 
particular tracking style desired. The processed 
error signals, PVERR and PHERR are converted into DC 
motor drive voltages by identical horizontal (i.e., 
pan) and vertical (i.e., tilt) motor driver circuits 
32 and 34, respectively which drive the tilt and pan 
motors in the proper direction and at the proper 
speed to keep the camera pointed at the subject. 
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Smart Tracking Processes 

A typical automated tracking system utilizes the 
elementary two step process illustrated in FIG. 4, 
to eliminate any errors between position of a 
tracked subject and the camera direction. First, 
the tracking system measures the angular error 
between the tracked subject and the tracking 
device/camera, as indicated by block 70 in FIG. 4. 
Next, the error information is used to generate a 
correction signal to control the motor speed and 
motion of the tracking device/camera, as indicated 
by block 72. The new position of the tracking 
device/camera is then utilized in calculating future 
angular errors. 

In the present invention, a number of signal 
processing techniques for adjusting the response 
characteristics of the tracking system give it 
attributes that result in videos that are pleasing 
to a human viewer. In a first embodiment, error 
signals VERR and HERR are manipulated by digital 
processor 46 in accordance with a 2-bit, 
user-supplied, STYLE signal, provided via buss 60, 
which indicates the desired style of tracking. 

The STYLE signal selects a set of parameters 
stored in a digital processor 46 or in a separate 
semiconductor memory. The parameters comprise a 
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number of data variables which represent acceptable 
system limitations for the selected style or mode of 
tracking, as well as variables representing real 
time data about the position of the tracked 
subject. The set of system parameters are used by 
processor 46 to perform a series of calculations to 
adjust the values of error signals VERR and HERR . 
The processed signals PVERR and PHERR are supplied 
over busses 62 and 64, respectively, to control 
motor drives 32 and 34 , respectively, which, in 
turn, cause camera 10 to point to the subject as 
closely as possible within the constraints that the 
resulting motion be pleasing to the eye of the 
viewer . 

In a second embodiment, the user-supplied STYLE 
signal describing the desired tracking style is 
replaced with a signal generated by digital 
processor 46, based on the past history of the 
tracked subject's motions as described by the most 
recent values of some of the variables . 

FIGS. 5 and 7 illustrate the processes executed, 
according to the first and second embodiments, 
respectively, by digital processor 46 to process 
signals VERR and HERR. The processes adjust the 
response characteristics of the tracking system to 
give it attributes that result in videos that are 
pleasing to a human viewer. Each of these 
algorithmic processes, represented by blocks 78, 80, 
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82, 84, 86, 88, 90 and 92, are described in detail 
below. 

In the techniques of the present invention the 
manipulation of the vertical error signal VERR is 
essentially identical to that of horizontal error 
signal HERE.. Accordingly, the concepts of the 
present invention will be explained only in regard 
to the manipulation of the error signal HERR. 

According to a first embodiment of the present 
invention the user/operator of the tracking system 
selects the desired style or mode in which to track 
a subject. This selection is done upon powering the 
system, as indicated by block 78 of FIG. 5. The 
user communicates the selected mode to digital 
processor 46 with the two-bit STYLE signal, supplied 
over line 60. Optionally, processor 46 may be 
programmed to select a default set of parameters 
representing the initial tracking mode or style upon 
power up. Upon the selection of the desired mode, 
processor 46 sets all relevant variables to the 
appropriate values associated with the particular 
style of tracking selected. 

The process of measuring the horizontal and 
vertical angular errors between the position of the 
tracked subject and the direction in which the 
tracking device/camera is pointing, as represented 
by block 80 of FIG. 5, is described above and 
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entails those functions to the point where error 
signals VERR and HERR are presented to digital 
processor 46. Likewise, the conversion of 4-bit 
offset binary signals and into DC voltages to 
control motors 42 and 44, respectively, as 
represented by block 90 of FIG. 5 , : is described in 
detail in U.S. Patent Application Serial No. 
0 7/396,987, now allowed, and will not be described 
further. The algorithmic steps represented by 
blocks 82, 84, 86 and 88 are performed by digital 
processor 46 in conjunction with the STYLE signal 
and are described in detail below. 

Deadzone Controller 

A first process as executed by digital processor 
46 to produce a tracking style which is more 
pleasing to human viewers includes a deadzone 
controller, represented by process block 82 in Fig. 
5 and implemented with the algorithmic steps 
illustrated in Fig. 6A. Unlike existing tracking 
systems where the desire to keep the subject 
centered at all times within the camera frame takes 
precedence over other esthetic goals, the present 
invention allows the subject to move anywhere within 
a central portion of the camera frame referred to as 
the "deadzone", without adjusting the camera 
position . 

The deadzone controller is implemented with a 
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number of variables, stored in digital processor 46, 
which represent various system parameters. The 
value of these parameters varies according to the 
desired tracking style specified by the user. 
Processor 46 manipulates the variables to achieve 
the desired controller function. 

The deadzone is described by a pair of angular 
values (DEAD LEFT , DEAD RIGHT) that correspond to 
the range of angular error that is rolerated in each 
direction before the camera position is readjusted. 
Initially, the deadzone is symmetric about the 
center of the subject and is set to (- THETA, 
+ THETA) , where the variable THETA is an angular 
value and is a parameter of the tracking context. 
The total angular range of the deadzone is equal to 
twice the value of THETA. 

The ERROR variable represents the value of the 
angular ERROR along one axis between the tracked 
subject and the direction of the tracking 
device /earner a. The deadzone controller takes the 
value of the ERROR variable and remaps the variable 
into a new ERROR variable which accounts for the 
deadzone. For the purposes of this exemplary 
embodiment, the ERROR variable will represent the 
angular error on the horizontal axis HERR, however 
it could eguivalently represent the angular error 
along the vertical axis, VERR. 
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Referring to FIG . 6A, the deadzone is 
represented by the values (DEAD LEFT, DEAD RIGHT). 
Digital processor 4 6 then determines whether the 
value of ERROR is still within the deadzone, as 
indicated by decisional step 96. If the value of 
the ERROR variable is within the deadzone, the ERROR 
variable is set to zero, as indicated by processing 
step 94. When the value of ERROR variable is not: 
within the deadzone, the subject has moved out of 
either side of the deadzone. 

Once it is determined that the value of ERROR 
is not within the deadzone, as determined by step 
96, the direction of the ERROR is determined, as 
indicated by decisional step 98. If the value of 
ERROR is to the left of the deadzone, the value of 
DEAD LEFT is subtracted from the value of ERROR, as 
indicated by processing step 100. Conversely, if 
the value of ERROR is to the right of the deadzone, 
the value of DEAD RIGHT is subtracted from the value 
of ERROR, as indicated by processing step 102. In 
this manner, as the subject moves out of either side 
of the deadzone the angular error is transformed by 
subtracting the edge of the deadzone so that it 
describes how far in degrees from the edge of the 
deadzone the .subject is rather than how far from the 
center of the camera frame. 

Ultimately, the deadzone may be implemented as a 
zone of reduced response. In this implementation 
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the response to the system, i.e. the relation 
between ERROR and MOTOR SPEED, in the reduced 
response zone can be set to a very low gain rather 
than a gain of zero, This implementation allows the 
system to slowly recenter the camera on the tracked 
subject instead of not moving at all as in the case 
with the deadzone implementation. However, such an 
implementation requires very close control over the 
motors 42 and 44 at low speeds, making the 
implementation practical only with precision 
stepping motors and not typical analog motor 
controllers. In addition, such an implementation 
complicates the processing performed by the 
directional continuity controller and motion 
controller, as explained hereinafter. 

Deadzone Shift Controller 

A second process executed by digital processor 
46 comprises a deadzone shift controller, 
represented by block 84 in Fig. 5 and implemented by 
the algorithmic steps illustrated in Fig. 6B. As a 
subject moves out of the deadzone and the camera 
begins to track the subject, the deadzone shifts. 
The purpose of the deadzone shift is to allow the 
earner., to catch-up with the moving subject rather 
than always lagging by THETA. Before the deadzone 
can be shifted, a number of limits are initialized. 
The variable MAXDEAD represents the value of THETA 
plus t*ie maximum number of degrees that one edge of 
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the deadzone may be shifted toward the center of the 
camera frame. The MAXDEAD value is a function of 
motor speed. The motor speed for which MAXDEAD is 
greater than THETA , its minimum value, is the point 
at which the deadzone is allowed to shift. The 
motor speed for which MAXDEAD is equal to (2 • 
THETA) is the point at which the deadzone has 
shifted sufficiently for the tracking system to 
begin leading the subject. 

The variable DEADSH1FT represents the maximum 
rate in degrees per second that the deadzone may be 
shifted, and is likewise a function of the motor 
speed. The value of DEADSHIFT is typically some 
small fraction of the motor speed. 

Before shifting the deadzone, the values of the 
variables of MAXDEAD and DEADSHIFT are set to a 
function of the value of MOTOR SPEED , which 
represents the current value of the speed of the 
motor 44, as indicated by processing step 104. 

The shift of the deadzone occurs in the 
following manner.- The value of ERROR is compared to 
the right edge of the deadzone, DEAD RIGHT, as 
indicated by decisional step 106. If the value of 
ERROR is beyond the right edge of the deadzone, the 
deadzone is shifted left, as indicated by processing 
step 108. This leftward shifting is accomplished by 
subtracting the value of DEADSHIFT from both the 
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value of DEAD LEFT and the value of DEAD RIGHT. If 
the value of ERROR is not beyond the right edge of 
the deadzone, a comparison is made to determine 
whether the ERROR value is beyond the left edge of 
the deadzone, as indicated by decisional step 110. 
If the value of ERROR is beyond the left edge of the 
deadzone, the deadzone is shifted to the right, as 
indicated by processing step 112. This rightward 
shifting of the deadzone is accomplished by adding 
the value of DEAD SHI FT to both the value of DEAD 
RIGHT and the value of DEAD LEFT. 

The new values of DEAD LEFT and DEAD RIGHT now 
represent the shifted deadzone which is then 
compared to the value MAXDEAD to determine if the 
deadzone has been shifted beyond the allowable limit 
of MAXDEAD. To accomplish this, the value of DEAD 
LEFT is compared to the value of —MAXDEAD , as 
indicated by decisional step 114. If the left edge 
of the deadzone is beyond the allowable limit, i.e. 
DEAD LEFT < —MAXDEAD , the deadzone is shifted to the 
left limit. This is accomplished by setting DEAD 
LEFT to -MAXDEAD and setting DEAD RIGHT equal to 
(2 • THETA - MAXDEAD), as indicated by processing 
step 116. 

If the left edge of the deadzone is within the 
allowable limit, the right edge of the deadzone, 
DEAD RIGHT, is compared to the value of MAXDEAD, as 
indicated by the decisional step "118. If the right 
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edge of the deadzone is beyond the allowable limit, 
the deadzone is shifted to the right limit by 
setting DEAD RIGHT equal to MAXDEAD and setting DEAD. 
LEFT equal to (MAXDEAD - 2 • THETA) , as indicated 
by processing step 120, 

To exemplify the above process, assume that the 
subject moves out of the right side of the deadzone 
and the camera begins to track the subject. Since 
the camera is in motion the value of MAXDEAD will be 
greater that THETA , its minimal centered value, 
allowing the deadzone to shift in the direction 
opposite to the direction of the camera's motion. 
If THETA is given a value of 5°, the initial 
deadzone is (-5, +5) degrees. With a current motor 
speed of 25° per second, typical values could be 9° 
for the maximum allowable deadzone value, MAXDEAD , 
and 1° per second for the maximum rate of deadzone 
shift, DEADSHIFT. Over a period of 4 seconds, the 
deadzone will shift from (-5, +5) to (-9, +1). At 
this point, the camera will be trailing the subject 
by 1° rather than the 5° that would result if the 
deadzone had not been shifted. 

As, the camera moves at a faster speed the limit 
of the deadzone, MAXDEAD , will eventually shift to 
greater than 10° and the deadzone will begin to 
shift to (< -10, <0), with the effect of leading the 
tracked subject. As the subject slows down, the 
maximum deadzone, MAXDEAD, will decrease again to 
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5°, and the deadzone will begin to shift back 
towards its normal rest value of (-5, +5) which 
corresponds to its normal centered position, 
allowing the subject to move within the symmetric 
deadzone in the camera frame. 

The size of the deadzone, (2 • THETA) ; the 
function controlling the rate at which the deadzone 
is shifted, DEADSHIFT; and the function controlling 
the amount the moving subject is led by the camera, 
MAXDEAD, are all parameters that may be adjusted 
depending upon the tracking strategy, as specified 
by the user. Note that the choice of the DEADSHIFT 
function effects the overall stability of the 
control servo-loop of the system and should be 
chosen to avoid instability, as can be appreciated 
by one skilled in the art. 

Continuity Control 

A third process executed by digital processor 46 
to produce a tracking style which is more pleasing 
to human viewers includes a directional continuity 
controller, represented by process block 86 in Fig. 
5 and implemented by the algorithmic steps 
illustrated in Fig. 6C. Human viewers are very 
sensitive to changes in the direction of the motion 
of an image. Accordingly, it is important to 
minimize the number of times the direction of camera 
motion changes. Viewers are particularly sensitive 
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to directional changes that occur either at low 
speeds or when the tracking device/camera moves from 
high speed motion to a stop and reverses direction. 
The directional continuity controller guarantees 
that there is not a series of directional changes 
that would be annoying to the viewer. Like the dead 
zone and dead zone shift controllers, the 
directional continuity controller is implemented 
with a number of variables, stored in digital 
processor 46 and representing various system 
parameters. The values of these parameters vary 
according to the desired tracking modes specified by 
the user. Processor 46 manipulates the variables to 
achieve the desired controller function. 

The variables relevant to the directional 
continuity controller are as follows. The LAST 
DIRECT variable represents, -the sign of the 
direction in which the camera was last moving. The 
ERROR DIRECT represents, the sign of the angular 
error between the camera direction and the subjects 
position, i.e., the direction the camera must move 
to close-in on the subject. The STABLE TIME 
variable represents, in seconds, the elapsed time 
since the last motion of the camera. The REVERSE 
DELAY variable represents the minimum time that the 
subject is required to move in a new direction 
before the camera will begin to move. Until the 
time specified by the REVERSE DELAY variable is met 
the camera is not allowed to change direction. Note 
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that some portion of the REVERSE DELAY time will be 
spent as the subject crosses the dead zone of the 
camera frame. 

The algorithmic steps executed by the 
directional continuity controller are illustrated in 
Fig. 6C. At any point in time, the LAST DIRECT and 
STABLE TIME variables represent the last direction 
of camera motion and the elapsed time since such 
motion, respectively. If other control processes 
try to move the camera in the same direction, such 
motion is allowed and the elapsed time is reset. To 
accomplish this, processor 46 compares the value of 
the LAST DIRECT variable to zero, as indicated by 
decisional step 122. If the value of LAST DIRECT 
equals zero, indicating that the camera has not 
recently been in motion, the camera is allowed to 
move in the direction of the error, as indicated by 
processing step 126. During step 126 the value of 
the STABLE TIME variable is set to zero and the 
value of the LAST DIRECT variable is set to the 
value of the ERROR DIRECT variable. 

If in decisional step 122, the LAST DIRECT 
variable is not equal to zero, indicating that the 
camera has recently been in motion, the value of the 
ERROR DIRECT variable is compared to the LAST DIRECT 
variable, as indicated by decisional step 126. If 
the ERROR DIRECT variable equals the LAST DIRECT 
variable, indicating that the error direction and 
the last camera direction are the same, processing 
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step 126 is executed as previously described. 

If the error direction and the last camera 
direction are not equal, the elapsed time since the 
last motion, STABLE TIME , is compared to the REVERSE 
DELAY constant, as indicated by procedural step 
128, If the value of the STABLE TIME variable is 
greater than or equal to the REVERSE DELAY value , 
then processing step 126 is executed as previously 
described. If however, the elapsed time since the 
last motion is less than the time specified in the 
REVERSE DELAY variable the camera is not allowed to 
reverse directions, as indicated by processing step 
130. In processing step 130 the value of the STABLE 
TIME is incremented and the ERROR variable is set to 
zero. The value of the LAST DIRECT variable remains 
unchanged . 

In this manner, the directional continuity 
controller allows the camera to move in the 
direction of its most recent motion. However, a 
minimum threshhold of time is required before the 
camera is allowed to reverse directions. The value 
of the time delay specified in the REVERSE DELAY 
variable is a parameter of the tracking context, as 
indicated by the user-specified STYLE signal. 
Typical ranges for the value of the REVERSE DELAY 
variable can be from 1/6 of a second to greater than 
5 seconds, without causing instability in the servo 
loop of the tracking system. 
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Motion Controller 

A fourth process executed by digital processor 
46 includes a motion controller, represented by 
process block 88 in Fig. 5 and implemented by the 
algorithmic steps illustrated in Fig. 6D. 

In most video pictures, the background typically 
represents a greater portion of the picture than 
does the subject. If the motion of the scene 
against the background is jerky, the picture will be 
disconserting to the viewer, even if this results 
from tracking the inconsistent motions of the 
subject. The motion controller performs a pseudo 
filtering function in the response of the system by 
selecting acceleration time constants that are 
different from the deceleration time constants for 
the motors driving the camera. As a result, the 
camera motion starts smoothly while avoiding any 
tendency to overshoot the subject when the motion is 
halted. in addition to the filtering function, the 
motion controller places limits on the maximum 
velocity and acceleration of the camera depending on 
the context of the subject's motion. As with the 
other process controllers, the motion controller is 
implemented with a number of variables, stored in 
digital processor 46 and representing various system 
parameters. The value of these parameters varies 
according to the desired tracking modes selected by 
the user . 
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The variables relevant to the motion controller 
are as follows. The variable ERROR is defined as 
similar to that used by the dead zone controller. 
The variable LASTERROR represents the previous value 
of ERROR. The variable MAXACC represents the 
maximum allowable acceleration, measured in degrees 
per second squared, for the selected tracking mode. 
The variable MAXVEL represents the maximum allowable 
velocity, measure in degree per second, for the 
selected -tracking mode. The variable K ACC 
represents the exponential smoothing time constant 
that controls the rate of which the motor 44 is 
accelerated. The value (1 - K ACC) is the fraction 
of the exponentially averaged history of the signal 
that is left after one time step. The variable K 
DEACC represents the exponential smoothing time 
constant that controls the speed at which motor 44 
is decelerated. The value ( 1 — K DEACC) is the 
fraction of the exponentially averaged history of 
the signal that is left after one time step. 
Typically the variable K DEACC is greater than or 
equal to the variable K ACC to make the acceleration 
as smooth as possible while avoiding overshooting of 
the subject when the camera motion stops. The 
values of both K ACC and K DEACC are in the range 
between zero and one. 

The motion controller operates as follows. If 
the value of ERROR would lead to a camera velocity 
that is greater than the maximum allowable velocity 
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prescribed by the tracking mode, the velocity is 
limited. This is accomplished by comparing the 
value of ERROR to the value of MAXVEL, as indicated 
by decisional step 132. If the value of ERROR is 
greater than value of MAXVEL, the velocity is 
limited by setting ERROR equal to the value of 
MAXVEL, as indicated by procedural step 134. If the 
value of ERROR is not greater than MAXVEL in step 
132 it is compared to the value of —MAXVEL , as 
indicated by decisional step 136. If the value of 
error is less than -MAXVEL, the velocity is again 
limited by setting ERROR equal to —MAXVEL , as 
indicated by process step 138. 

If the difference between the current value of 
ERROR and the previous value of ERROR, LAST ERROR 
(which is the acceleration) is greater than the 
maximum acceleration prescribed by the tracking 
context, the camera velocity is limited to the 
current camera velocity plus the maximum 
acceleration. To accomplish this, the value of 
ERROR is compared to the sum of the LAST ERROR and 
MAXACC, as indicated by decisional step 140. If the 
value of ERROR is greater in step 140, the 
acceleration is limited by setting the value of 
ERROR to the sum of LAST ERROR and MAXACC, as 
indicated by process step 142. Otherwise, the value 
of ERROR is compared to the current velocity minus 
the maximum allowable acceleration, as indicated by 
decisional step 144. If the value of ERROR is less 
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than the value of LAST ERROR minus MAXACC, the 
* acceleration is again limited by setting the value 
of ERROR to LAST ERROR minus MAXACC, as indicated by 
processing step 146. 

After any acceleration or velocity limiting, 
digital processor 46 determines whether the camera 
is accelerating, as indicated by decisional step 
148. If the camera is accelerating, a smoother 
acceleration is achieved as indicated by process 
step 150. A smooth acceleration is achieved by 
computing a smooth error value, NSERROR using. the 
value of ERROR and the value of the last smoothed 
error, LSERROR, with Eq. (1) below. 

NSERROR = (1-K ACC) • LSERROR + K ACC • ERROR 
Eg. (1) 

Process step 150 is the equivalent of low pass 
filtering. The new smooth error, NSERROR, of Eq. (1) 
is then used to control the speed of motor 44. 

If the camera is not accelerating, processor 46 
determines if the camera is decelerating, as indicated 
by decisional step 152. If the camera is decelerating, 
a smooth decelaration is performed as indicated by the 
processing step 154. The smooth deceleration is 
achieved by computing a smooth error value, NSERROR 
using the value of ERROR and the value of the last 
smoothed error, LSERROR with Eq. (2) below. 
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NSERROR = (1-K DEACC) • LSERROR + K DEACC • ERROR 
Eg. (2) 

The above pseudo low pass filtering technique uses 
different time constants when accelerating and 
decelarating. The sensitivity of viewers to the 
smoothness of acceleration is greater than the 
sensitivity to the smoothness of decelarat ion . In a 
preferred embodiment the ratio between K ACC and K 
DEACC is typically 1:2. 

In the first embodiment of the present invention 
the user of the tracking system specifies a tracking 
context so that the system performance is specifically 
tuned for the context of the subject's motion. A 
typical selection of modes or tracking styles may 
include slow, general and fast action modes. A slow 
action mode, good for lectures, walking sports, playing 
with young children or any activity without running, is 
be characterized by slow, smooth motion with a tendency 
to lag the subject if the subject engages in a quick 
motion. A general action mode, good for general play 
is appropriate for smooth motion with a tendency to lag 
the subject only when the subject is at a full run. A 
fast action mode, good for tennis, basketball, soccer 
or any activity with full speed running and quick 
changes of motion would be appropriate for quickly 
tracking even the fastest moving subject and has a 
tendency to react more quickly than is visually 
pleasing if the action isn't fast enough to mask it. 
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Automatic Contexts Determination Controller 

According to the second embodiment of the 
present invention, the user-supplied STYLE signal, 
describing the desired tracking style, is replaced 
with a signal generated by digital processor 46, 
based on the past history of the tracked subject's 
motion as described by the most recent values for 
some of the system variables, as explained 
hereinafter . 

Figure 7 illustrates processes executed by 
digital processor 46 to process signals VEKR and 
HEKR. In this embodiment, processor 46 
automatically adjusts the previously described 
processes so that the response characteristics of 
the tracking system give it attributes that result 
in videos that are pleasing to the human viewer. 
Referring to Figure 7, the algorithmic process 
represented by blocks 78, 80, 82, 84, 86, 88, and 90 
are similar to those described with reference to 
Figure 5 and the first embodiment of the present 
invention. 

A fifth process executed by digital processor 46 
includes an automatic context controller, 
represented by process block 92 in Figure. 7 and 
implemented by the algorithmic steps illustrated in 
Figure 8. 
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Since a part of the function of the tracking 
system is to determine the position of the subject 
being tracked, there is a significant amount of 
information available that can be used to predict 
the style of future motions. Typically, the real 
activities that the subject engages in are quite 
stable in their range of motions. In the second 
embodiment of the present invention the parameters 
of the subject's motion are stored and then compared 
with the parameters expected to be associated with 
such a tracking context. The extracted parameters 
include: the spectral characteristics of the motion, 
the percentage of time the current context tracks 
properly, the maximum angular acceleration of the 
subject, and the minimum angular speed of the 
subject. These parameters are chosen more for ease 
of calculation than as being the best parameter. 

The variables relevant to the automatic context 
determination controller are as follows. The 
CONTEXT variable is the equivalent to the STYLE 
signal and designates the currently selected 
tracking context or mode. The MISSTRACKING variable 
represents the length of time the subject is outside 
of the dead zone. The MAXMISSTRACK variable 
represents the maximum allowable percentage of time 
t:.e subject may be allowed outside the dead zone for 
the particular tracking style selected by the 
CONTEXT variable. The AVG MOVE variable represents 
the average time the subject moved in a single 
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direction. The MOTOR SPEED variable is the same as 
previously defined regarding the first embodiment. 
The DELTA ERROR variable represents the change in 
angular error, in degrees, since the angular error 
of the subject was last measured. DELTA ERROR is 
added to the MOTOR SPEED to give the speed of the 
subject, CURRENT SPEED. The DELTA SPEED variable 
represents the change in CURRENT SPEED , in degrees 
per second, since the angular error of the subject 
was last measured. The MAX SPEED variable 
represents the maximum angular speed of the 
. subject. The MAX ACC variable represents the 
maximum angular acceleration of the subject. 

Following processing by motion controller 88 , 
the automatic context determination controller 92 
determines whether the current tracking context is 
appropriate for the motion of the subject. 
Referring to Figure 8, with each new value of ERROR, 
the amount of time in which the current system mode 
mistracked the subject is computed, as indicated in 
processing step 160. The absolute value of the 
ERROR variable is used to determine whether the 
subject is currently being mistracked. If the 
absolute value of the ERROR variable is less than or 
equal to a maximum allowable error, the value of 
MISSTRACKING is not incremented, indicating that the 
subject is still moving within the acceptable zone. 
However, if the absolute value of the ERROR variable 
is greater than the maximum allowable angular error, 
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the value of MI SSTRACKING is incremented, indicating 
that the subject has left the acceptable zone. 

The percentage of the time the system is 
mistracking is calculated by dividing MISSTRACKING 
by the total time, and compared to the maximum 
allowable percentage for the current mode, 
MAXMISSTRACK, as indicated in decisional step 162. 
If the percent of time the system is mistracking the 
value of MISSTRACK is greater than the value of 
MAXMISSTRACK, the value of the CONTEXT variable is 
set to the next faster contest or tracking mode, as 
indicated by processing step 164. Next the maximum 
angular speed of the subject, MAX SPEED, is 
calculated as indicated by processing step 166. 
This is done by setting the value of MAX SPEED to 
the maximum of the last value of MAX SPEED and the 
absolute value of the current speed of the subject, 
CURRENT SPEED, which is computed as in Eg. (3) below. 

CURRENT SPEED = (MOTOR SPEED + DELTA ERROR) Eg. (3) 

The absolute value of the speed of the subject 
is used so the value of MAX SPEED will be direction 
independent . 

Next, the maximum angular acceleration of the 
subject, MAX ACC, is calculated as indicated by 
processing block 168. This is done by setting the 
value of MAX ACC to the maximum of the last value of 
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MAX ACC and the absolute value of the current 
acceleration of the subject, CURRENT ACC, which is 
computed as in Eg. 4 below 

CURRENT ACC = DELTA SPEED Eg. (4) 

The absolute value of the difference in speed of 
the subject is used so the value of MAX ACC will be 
direction independent. 

Finally, the average time of the subject's 
movement is computed as indicated by processing step 
170. The average time is calculated by simply 
computing the average time between directional 
changes and setting the AVG MOVE variable to such 
average. 

Associated with each available context or mode 
is a range of values that correspond to appropriate 
ranges of motion of the subject. A determination is 
made whether the current values characterizing the 
subject's motion more closely match those of a 
slower context, as indicated by decisional step 
172. If the characteristics of the subject's motion 
will closely match a slower mode or context, the 
value of CONTEXT is set to the next slower context, 
as indicated by processing step 174. Otherwise, the 
tracking context remains the same. 
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Variable Ranges 

Depending on the actual physical implementation 
of the tracking system, various other variables may 
be relevant. For instance, a GAIN variable may be 
necessary for controlling the overall gain of the 
servo-loop in the tracking system. A MOTOR OFFSET 
value would allow the system to account for the 
minimum start up voltages required when utilizing 
analog motors. By adding a MOTOR OFFSET to the 
ERROR value during the motor control process the 
zero-crossing response of such a motor can be evened 
out- Setting the MOTOR OFFSET variable to zero has 
the effect of adding to the dead zone in a way which 
is difficult to compensate for with other variables 
in the system. 

Since the entire tracking system disclosed in either 
embodiment of the present invention forms a single 
servo system, it is difficult to give actual values 
for the variables contained in each different mode 
parameter set. Such parameter values are heavily 
dependent on the specific considerations of the 
entire electro-mechanical system. However, below 
are listed a number of ratios which represent a 
reasonable range of values for select variables 
between a slow action tracking mode and a fast 
action tracking mode. The overall gain of the 
tracking system, i.e., the response of MOTOR SPEED 
to ERROR, may range over a factor of 4:1. The ratio 
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of K ACC to K DEACC may range over a factor of 1:3. 
The values of K ACC and K DEACC may each range over 
a factor of 3:1. The REVERSE DELAY time may range . 
from 1/6 second to 6 seconds. Finally, the value of 
THETA may range over a factor of 5:1. 



Alternate Embodiments 

Although the present invention has been 
described with reference to a sampled, ultrasonic 
tracking system which uses DC motors, it will be 
obvious to one skilled in the art that the present 
invention is equally applicable to any tracking 
system . 

Such systems may be active, as in the exemplary 
embodiments, or may be passive tracking systems, 
such as infrared tracking systems which do not 
require a transmitter to track a subject. Other 
such tracking systems include systems which use 
frame to frame motion comparison and pattern 
recognition techniques to identify portions of an 
image that are to be maintained within the camera 
frame. Still other systems may be sampled, as in 
the exemplary embodiments, or may be continuous, and 
may use infrared, radio frequency, or sonic 
transmission to communicate between the 
transmitter/receiver. Other systems may use 
precision stepping motors to control the camera or 
tracking device instead of analog motor controllers. 
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In the above exemplary embodiments, the 
algorithmic processes represented by blocks 78, 80, 
82, 84, 86, 88, 90 and 92 are implemented with. a 
plurality of system variables and a programmed 
digital processer. It will be obvious to those 
reasonably skilled in the art that functionally 
equivalent processes may be achieved with an 
implementation using small, medium or large scale 
logic components as well as analog circuit 
elements. Such hardware implementations are well 
within the scope of those reasonably skilled in the 
art and are within the scope and spirit of the 
present invention . 

The above invention has been described with 
reference to the horizontal angular error signal, 
HERR, which represents one axis of a multiple axis 
system. It is obvious to those skilled in the art 
how the control functions disclosed herein are 
eqiially applicable to the vertical angular error 
signal, VERR, provided the designer accounts for the 
differing sensitivity of the human eye to vertical 
axis motion. It will be further obvious to those 
reasonably skilled in the art that digital processor 
46 may perform the control functions for both the 
horizontal and the vertical angular error signals, 
as shown in FIG. 3, or that two such digital 
processors may be used, one for each axis of motion. 
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Further, since many activities have different 
motions in each axis, contemplated tracking systems 
may allow different tracking modes to be assigned to 
each axis. Conversely, a contemplated tracking 
system may provide for automatically or 
user-selected tracking modes which fix the 
horizontal and vertical axis tracking styles to each 
other, the parameters of such modes being defined 
for a specific two-axis activity. 

It should be further understood that the 
concepts of the present invention apply to any 
tracking system, which given a set of criteria, uses 
the particular parameters of the tracking system to 
optimize the system performance to satisfy the 
criteria. The exemplary embodiment listed a number 
of visual criteria. It will be obvious to those 
reasonably skilled in the art that other criteria 
associated with the smart tracking of a subject, but 
not disclosed herein may be similarly achieved with 
the apparatus and methods disclosed in the present 
invention, given the proper system parameters. Such 
criteria and the means for optimizing the tracking 
system performance to meet the criteria are within 
the scope of the present invention. 

Accordingly, it will be appreciated that the 
detailed disclosure has been presented by way of 
example only and is not intended to be limiting. 
Various alternative, modifications and improvements 
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will readily occur to those skilled in the art, 
may be practiced without departing from the spir 
and scope of the invention. The invention is 
limited only as required by the following claims 
equivalents thereto. 

What is claimed is: 
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1. A system for permitting a device to 
point at and track the movements of a moving 
subject, comprising: 

a. a signal source for generating signals 

which can be tracked; 

b. control means for receiving signals 
from the source and for generating relative 
orientation data therefrom in the form of control 
signals which vary as continuous analog functions 
with changes in relative orientation; 

c. means for storing plural sets of data 
values, each of said data value sets representing 
Selected values for processing said control signals 
in accordance with the anticipated motion of the 
subject; 

d. means for selecting one of said sets of 

data values; 

e. means for processing said control 
signals in accordance with the one of said sets of 
data values selected by said selecting means to 
produce processed control signals; and 

f. a drive unit for moving the device about 
at least one axis in response to said processed 
control signals to maintain the device pointed at the 
subject. 

2. The system of Claim 1 wherein each of said 
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data value sets comprises data values representing 
selected acceleration rates of the device and 
selected deceleration rates of the device, for a 
particular range of anticipated motion by the subject. 

3. The system of Claim 1 where each of said data 
value sets comprises data values representing a 
selected angular velocity of the device, and a maximum 
allowable angular error between the axis of the device 
and the subject, for a particular range of anticipated 
motion by the subject. 

4. The system of any of Claims 1, 2 or 3 wherein 
said means for selecting comprises a user-defined 
signal for indicating which of said plural sets of 
data values is to be selected. 

5. The system of any of Claims l, 2, or 3 wherein 
said means for selecting comprises means for 
automatically selecting one of said plural sets of 
data values in anticipation of future relative motion 
of the subject based on past motion of the subject. 

6. A method of intelligently and automatically 
tracking a moving subject with a camera comprising the 
steps of: 

a. determining a last direction in which the 
camera moved about an axis; 

b. calculating an elapsed time value representing 
the elapsed time interval since the camera last moved; 
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c. determining an error direction representing 
the present orientation of the subject relative to the 
camera; 

d. comparing said last direction with said error 
direction; 

e. moving the camera about the axis in the error 
direction if said last direction and said error 
direction are equal; 

f . comparing said elapsed time value with a 
predetermined value if said last direction and said 
error direction are not equal; and 

g. moving the camera about the axis in the error 
direction if the elapsed time value at least equals 
the predetermined value. 

7. The method of claim 6 further comprising the 
step of: 

h. maintaining the current angular orientation 
of the camera if the elapsed time value is less than 
the predetermined value . 

8. The method of claim 6 wherein said 
predetermined value is selected to represent the 
average time between directional changes by a subject 
for a predetermined class of motions. 

9. An apparatus for intelligently and 
automatically tracking a moving subject with a camera 
comprising : 

a. means for determining an error direction 
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representing the present orientation of the subject 
relative to the camera; 

b. means for sensing a last direction in which 
the camera moved about an axis; 

c. means for calculating an elapsed time value 
representing the elapsed time interval since the 
camera last moved; 

d. means for comparing the last direction with 
the error direction, and for comparing the elapsed 
time value with a predetermined value if the error 
direction and the last direction are not equal; and 

e. a drive unit, responsive to said means for 
determining and said means for comparing, for moving 
the camera in the error direction about said one axis 
if the elapsed time value at least equals the 
predetermined value or the last direction equals the 
error direction. 

10. The apparatus of claim 9 wherein said means 
for determining comprises: 

control means for receiving source signals from 
the subject and for continuously generating therefrom 
control signals representing the angular error 
between the position of the subject and the camera 
along at least one plane. 

11. A method of intelligently and automatically 
tracking a moving subject within the frame of a 
camera comprising the steps of: 

a. moving the camera about an axis to bring the 
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subject within the camera frame; 

b. establishing predetermined limits for the 
angular velocity and angular acceleration of the 
camera about the axis; 

c. defining a central zone within the camera 
frame and substantially symmetrical about the center 
of the subject; 

d. moving the camera about the axis at rates not 
to exceed the predetermined limits to maintain the 
subject within the zone; 

e. calculating a first percentage representing 
the total time the subject was outside said central 
zone during a given time interval; 

f . calculating the maximum angular speed and 
maximum angular acceleration of the subject within 
said given time interval; and 

g. redefining said predetermined limits for the 
angular acceleration and angular velocity of the 
camera in accordance with the anticipated motion of 
the subject, 

12. The method of claim 11 wherein step (g) 
comprises comparing the first percentage with a 
predefined maximum allowable percentage and selecting 
new predetermined limits having increased values for 
the angular velocity and angular acceleration of the 
camera when the first percentage exceeds the 
established predetermined limits. 

13. The method of claim 11 wherein step (g) 
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comprises selecting new predetermined limits having 
decreased values for the angular velocity and angular 
acceleration of the camera when the maximum angular 
speed and maximum angular acceleration of the subject 
are less than the predetermined limits for the 
angular velocity and angular acceleration of the 
camera . 

14. An apparatus for permitting a camera to 
maintain a moving subject within the frame of a 
camera in a manner which is visually pleasing to a 
viewer, the apparatus comprising: 

a. means for storing a plurality of 
predetermined limits for the angular velocity and 
angular acceleration of the camera about an axis; 

b. a drive unit, responsive to said means for 
storing, for moving the camera about the axis at 
rates not in excess of the predetermined limits to 
maintain the subject within the zone; 

c. means for defining a central zone within the 
camera frame and substantially symmetrical about the 
center of the subject; 

d. first means for calculating a first 
percentage representing the total time the subject 
was outside the central zone during a given time 
interval; 

e. second means for calculating the maximum 
angular velocity and maximum angular acceleration of 
the subject within the given time interval; and 

f. means, responsive to said first and said 
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second means, for selecting new predetermined limits 
from the means for storing for the angular 
acceleration and angular velocity of the camera in 
anticipation of future motion of the subject. 

15. The apparatus of claim 14 wherein the means 
for selecting comprises means for comparing the first 
percentage with a predetermined maximum allowable 
percentage stored in said means for storing and for 
selecting new predetermined limits for the angular 
velocity and the angular acceleration of the camera 
when the first percentage exceeds the predetermined 
maximum allowable percentage. 

16. The apparatus of claim 14 wherein said means 
for selecting comprises means for comparing the 
maximum angular velocity and maximum angular 
acceleration of the subject with the current 
predetermined limits for angular velocity and angular 
acceleration of the camera, and for selecting new 
predetermined limits for the angular acceleration and 
angular velocity of the camera when the maximum 
angular velocity and maximum angular acceleration of 
the subject are substantially less than or 
substantially greater than the current predetermined 
limits . 

17. A method of intelligently and automatically 
tracking a moving subject within the frame of a 
camera comprising the steps of: 
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a. defining a central zone within the camera 
frame by a pair of angular coordinates along one axis 
and substantially symmetric about the center of the 
subject at a first time; 

b. determining an angular error along said one 
axis between the subject and the camera at a second 
time ; 

c. determining whether the angular error is 
within the central zone; 

d. determining an error direction representing 
the direction the camera must move to reduce the 
angular error at the second time, if the angular 
error is not within the central zone; 

e. moving the camera about said one axis in the 
error direction; and 

f . redefining the angular coordinates of the 
central zone so that the central zone is shifted in a 
direction which is opposite the error direction. 

18. The method of claim 17 wherein step (f) 
comprises the steps of: 

f.l subtracting the value of one of the angular 
coordinates of the central zone from the value of the 
angular error to form a difference value; and 

f.2 subtracting the difference value from each 
of the angular coordinates to form new angular 
coordinates . 

19. The method of claim 17 further comprising 
the step of : 
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g. maintaining the camera in its current 
orientation if the angular error is within the 
central zone. 

20. An apparatus for intelligently and 
automatically tracking a moving subject within the 
frame of a camera comprising: 

a. means for defining a central zone within the 
camera frame by a pair of angular coordinates along 
one axis and substantially symmetric about the center 
of the subject at a first time; 

b. means for determining an angular error along 
said one axis between the subject and the camera at a 
second time; 

c. means for determining whether the angular 
error is within the central zone; 

d. means for determining an error direction 
representing the direction in which the camera must 
move to reduce the angular error at the second time, 
if the angular error is not within the central zone; 

e. a drive unit for moving the camera about said 
one axis in the error direction; and 

f. means for redefining the angular coordinates 
of the central zone so that the central zone is 
shifted in a direction which is opposite the error 
direction. 

21. The apparatus of claim 20 wherein said means 
for redefining comprises: 

means for subtracting the value of one of the 



.BbiSDOCID: <WO 9204641 A 1 J_> 



WO 92/04641 



- 49 - 



PCT/US91/063I6 



angular coordinates defining the central zone from 
the value of the angular error to form a difference 
value; and 

means for subtracting the difference value from 
each of the angular coordinates to form new angular 
coordinates . 

22. An apparatus for permitting a camera to 
maintain a moving subject within the camera frame in 
a manner that is visually pleasing to a viewer, the 
system comprising: 

a. a subject for generating signals which can be 

tracked; 

b. control means for receiving signals from the 
subject and for continuously generating relative 
orientation data therefrom in the form of control 
signals, said control signals representing the 
angular error between the position of the subject and 
the camera along at least one plane; 

c. means for anticipating future relative motion 
of the subject based on past motion of the subject; 

d. means, responsive to said control signals, 
for determining whether the angular error represented 
by the control signals is within an acceptable range 
given anticipated motion by the subject, and for 
processing said control signals if said angular error 
is not within the acceptable range; and 

e. a drive unit, responsive to said processed 
control signals, for selectively moving the camera 
about an axis to maintain the subject within the 
camera frame. 
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SMA-RT TRACKING SYSTEM 



Related Applications 

This application is co-pending with United 
States Patent Application Serial No. 07/396,987, 
filed August 22, 1989, now United States patent 
4,980,871, entitled Ultrasonic Tracking System, the 
subject matter of which is incorporated herein by 
reference . 



Field of the Invention 

The present invention relates to measuring and 
tracking systems and, more particularly, to a system 
wherein the manner in which a camera tracks a moving 
subject is dependent upon the context and history of 
the subject's movements. 

Background of the Invention 

When an automatic tracking system, such as that 
disclosed in U.S. Patent 4,980,871, directs a video 
camera for human viewers, there are a number of 
aesthetic factors which should be considered. In 
particular, factors such as the centering of the 
subject within the camera frame, the smoothness of 
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the camera motion against a stable background, and 
the continuity of the direction of camera motion are 
all elements which make a video picture 
aesthetically pleasing to human viewers. 
Conversely, a video picture which is characterized 
by abrupt or continuous camera motion and 
inconsistent camera tracking speed has a tendency to 
violate a viewer's visual satisfaction criteria. 

A typical automated tracking system utilizes a 
two step process to eliminate any errors between the 
position of a tracked subject and the camera 
direction. First, the tracking system measures the 
angular error between the tracked subject and the 
orientation of the tracking device/camera. Next, 
the error information is used to generate a 
correction signal to control the motion of the 
tracking device/camera. The new camera position is 
then utilized in calculating future angular errors. 
The resulting tracking is mechanical with a tendency 
to be just as annoying to the viewer as if the 
tracking was very inaccurate. For example, constant 
readjustment of the camera position to every motion 
of the subject can cause rapid eye fatigue m the 
viewer . 

Additionally, typical automated tracking systems 
use the same tracking strategy and style for all 
subjects and contexts, whether the subject is giving 
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a lecture or participating in a fast-paced athletic 
event. The inability of such tracking systems to be 
set to, or alternate between, different styles 
severely limits their applications. 

Unlike an automated tracking system, a human 
cameraman directs a camera to a subject with less 
than complete accuracy. The cameraman further 
adjusts the tracking style according to the context 
in which the subject moves and the immediate past 
history of the subject's motions. The resulting 
video contains minor tracking inaccuracies which are 
visually more pleasing to a human viewer than the 
exacting and mechanical tracking of an automated 
system. 

Accordingly, it is an object of the present 
invention to provide a system which allows a camera 
to track a moving subject in a manner which is 
visually and aesthetically pleasing to human 
viewers, by selectively permitting angular errors 
between the position of the subject and the 
direction in which the tracking device/camera is 
pointing . 

Another object of the present invention is to 
provide a system which allows a camera to track a 
moving subject in a specific style which is 
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appropriate to the context in which the subject is 
moving . 

A further object of the present invention is to 
provide a system which allows a camera to track, a 
moving subject in a style which is determined by the 
system, based on the past history of the subject's 
motions . 

Summary of the Invention 

A system for permitting a device to point at and 
track the movements of a subject comprises a signal 
source and a control means which receives signals 
from the source and generates therefrom control 
signals. A memory means stores plural sets of data 
values, each of the sets representing the operating 
parameters of said system in a specific operational 
mode. Means are provided for selecting one of said 
plural sets of data values. A processor processes 
the control signals in accordance with the selected 
data values and produces processed control signals. 
A drive unit moves the device about an axis in 
response to the processed control signals to 
maintain the device pointed at the subject. 

In one embodiment, the means for selecting one 
of the plural sets of data values comprises a 
user-defined signal for indicating the desired data 
value set and its corresponding operational 
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parameters. In a second embodiment, the means for 
selecting a data value set is a function of the 
processor which, when processing the control 
signals, determines whether the selected data value 
set is appropriate for the moving subject. If not, 
the processor selects a new data value set and 
thereby changes the operational mode of the system. 

The forgoing and other features, objects and 
advantages of the invention will be more fully 
understood by reading the detailed description 
below, in conjunction with the drawing. 

Brief Description of the Drawing 

In the drawing: 

FIG. 1 is a pictorial illustration of the 
drive unit of a tracking system suitable for use 
with the present invention; 

FIG. 2 is a diagrammatic illustration of 
the subject, such as an athlete, wearing the 
transmitter unit of a tracking system suitable 
for use with the present invention; 

FIG. 3 is a functional block diagram of a 
drive unit according to the invention; 

FIG. 4 is a flow chart of the error 
correction process of prior art automatic 
tracking systems; 

FIG. 5 is a flow chart of the error 
correction process executed by the drive unit of 
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FIG. 3 according to a first embodiment of the 
present invention; 

FIG. 6A is a flow chart of the algorithmic 
steps for calculating the dead zone of the 
process of FIG. 5; 

FIG. 6B is a flow chart of the algorithmic 
steps for shifting the dead zone of the process 
of FIG. 5; 

FIG. 6C is a flow chart of the algorithmic 
steps for continuity control of the process of 
FIG. 5; 

F.IG. 6D is a flow chart of the algorithmic 
steps for motion control of the process of FIG. 
5; 

FIG. 7 is a flow chart of the error 
correction process executed by the drive unit of 
FIG. 3 according to a second embodiment of the 
present invention; and 

FIG. 8 is a flow chart of the algorithmic 
steps for automatic context determination of the 
process of FIG. 7. 

Detailed Description 

The present invention provides a smart tracking 
system which in an exemplary embodiment, is an 
active tracking system comprising two units. The 
first unit is a small, battery-operated, 
omni-directional ultrasonic transmitter which is 
carried by the subject to be tracked. The 
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transmitter produces brief, bursts of sound at 
frequencies above the range of human hearing. The 
second unit comprises a motorized camera drive 
capable of tilting and panning an attached camera. 
It should be understood that the present invention 
may also be implemented with a passive tracking 
system, not requiring a transmitter worn by the 
subj ect . 

In an exemplary embodiment, a battery-operated 
ultrasonic receiver is attached to the camera 
drive. The receiver is tuned to the same frequency 
as the transmitter carried by the subject. The 
receiver employs three ultrasonic microphones, 
together with electronic circuitry to detect and 
amplify the ultrasonic signals generated by the 
transmitter and intercepted by the microphones. The 
microphones are located at three of the four corners 
of an imaginary square centered on and perpendicular 
to the axis of the camera lens. The signals 
emanating from the subject will arrive at all three 
microphones simultaneously only if the camera is 
pointed directly at the subject. The receiver 
measures the relative arrival times of signals 
received by the microphone, determines the direction 
and degree to which the camera is not pointed 
directly at the subject, and causes a servo-system 
to drive motors to re-orient the camera (and 
receiver) to point toward the subject matter. This 
re-orientation or tracking is done in a manner that 
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is visually pleasing to human viewers, and is 
adaptable to a variety of contexts. 

Referring to FIG . 1, a television or 
motion-picture camera or other device 10 (hereinafter 
referred to as a camera, without loss of generality) 
is attached to a platform or mount which is part of 
a motorized drive unit, generally designated 12, 
that is in turn mounted to a tripod 14 or other 
support. Drive unit 12 can tilt camera 10 up and 
down about horizontal axis 16 from about 30° below 
the horizontal to about 30° above the horizontal. 
The drive unit can also pan the camera left and 
right about vertical axis 18 through repeated 360° 
circles. The drive unit responds to ultrasonic 
signals received via microphones 20A, 20B and 20C to 
control direct-current motors (not shown) to move 
the camera mount . 

A subject 22 as shown in FIG. 2, who is to be 
recorded (e.g., an athlete), is provided with an 
ultrasonic transmitter 24 to be worn or carried on 
his or her person. The transmitter unit may 
include, for example, an elastic belt 26 with buckle 
for wearing around the waist of the subject. 
Attached to the belt are three or more transducer 
units (only one transducer unit 28, being shown in 
the drawing) connected by flexible cables embedded 
in belt 26. 
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Drive unit 12 includes two direct-current motors 
42 and 44, respectively. Through conventional 
mechanisms, motor 42 tilts the camera mount up and 
down and the motor 44 pans the camera mount left and 
right. Since the camera is securely fastened to the 
mount, it undergoes the same motions. The motors 
are driven by electronic circuitry located on the 
circuit boards (not shown in FIG. 2) inside the 
drive unit housing. The circuitry is powered by 
batteries (also not shown). 

Three detector circuits 30A, 30B and 30C are 
shown in the block diagram of FIG. 3, one for each 
of the three ultrasonic transducers 20A, 20B and 
20C, respectively. Also shown are two motor driver 
circuits 32 and 34, one for each of the two DC 
motors 42 and 44, respectively. A controller 
circuit 36 and digital processor 46 operatively 
connect the detectors to the motor drivers. Each of 
the transducers 20A, 20B and 20C receive the 
ultrasonic bursts emitted by the transmitter unit 
and supplies a corresponding electrical signal to 
its associated detector circuit. The detector 
circuit detects the presence of an ultrasonic burst 
and produces a rectangular pulse whose leading edge 
coincides with the beginning of the detected burst. 
These pulses, 50A, 50B and 50C are routed to 
controller 36, which measures the delay between the 
pulses from the three detectors and loads digital 
error registers (not shown) with binary signals, 
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VERR and HERR. These 4-bit signals represent the 
angular error along the vertical and horizontal axes, 
respectively, between the camera direction and the , 
subject. VERR and HERR are updated each time an 
ultrasonic burst is received, 
i 

The structure and function of transmitter unit 
24, and drive unit 12, including direct-current 
motors 42 and 44, and the manner in which the drive 
unit 12 tracks the location of transmitter unit 24 
to keep the camera pointed at the subject, is 
discussed in detail in U.S. Patent 4,980,871, which 
is incorporated herein by reference, and will not be 
repeated for brevity. 

Digital processor 46 receives the VERR and HERR 
signals over busses 52 and 54, respectively. 
Processor 46, may be a microprocessor. One such 
microprocessor suitable for use with the present 
invention is the model MC 68705C8 available from 
Motorola, Inc., Phoenix, Arizona. Processor 46 
manipulates the values of VERR and HERR. in a manner 
to be described more fully below, according to the 
particular tracking style desired. The processed 
error signals, PVERR and PHERR are converted into DC 
motor drive voltages by identical horizontal (i.e., 
pan) and vertical (i.e., tilt) motor driver circuits 
32 and 34, respectively which drive the tilt and pan 
motors in the proper direction and at the proper 
speed to keep the camera pointed at the subject. 
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Smart Tracking Processes 

A typical automated tracking system utilizes the 
elementary two step process illustrated in FIG. 4, 
to eliminate any errors between position of a 
tracked subject and the camera direction. First, 
the tracking system measures the angular error 
between the tracked subject and the tracking 
device/camera, as indicated by block 70 in FIG. 4. 
Next, the error information is used to generate a 
correction signal to control the motor speed and 
motion of the tracking device/camera, as indicated 
by block 72. The new position of the tracking 
device/camera is then utilized in calculating future 
angular errors. 

In the present invention, a number of signal 
processing techniques for adjusting the response 
characteristics of the tracking system give it 
attributes that result in videos that are pleasing 
to a human viewer. In a first embodiment, error 
signals VERR and HERR are manipulated by digital 
processor 46 in accordance with a 2-bit, 
user-supplied, STYLE signal, provided via buss 60, 
which indicates the desired style of tracking. 

The STYLE signal selects a set of parameters 
stored in a digital processor 46 or in a separate 
semiconductor memory. The parameters comprise a 
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number of data variables which represent acceptable 
system limitations for the selected style or mode of 
tracking, as well as variables representing real 
time data about the position of the tracked 
subject. The set of system parameters are used by 
processor 46 to perform a series of calculations to 
adjust the values of error signals VERR and HERR. 
The processed signals PVERR and PHERR are supplied 
over busses 62 and 64, respectively, to control 
motor drives 32 and 34, respectively, which, in 
turn, cause camera 10 to point to the subject as 
closely as possible within the constraints that the 
resulting motion be pleasing to the eye of the 
viewer . 

In a second embodiment, the user-supplied STYLE 
signal describing the desired tracking style is 
replaced with a signal generated by digital 
processor 46, based on the past history of the 
tracked subject's motions as described by the most 
recent values of some of the variables. 

FIGS. 5 and 7 illustrate the processes executed, 
according to the first and second embodiments, 
respectively, by digital processor 46 to process 
signals VERR and HERR. The processes adjust the 
response characteristics of the tracking system to 
give it attributes that result in videos that are 
pleasing to a human viewer. Each of these 
algorithmic processes, represented by blocks 78, 80, 
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82, 84, 86, 88, 90 and 92, are described in detail 
below . 

In the techniques of the present invention the 
manipulation of the vertical error signal VERR is 
essentially identical to that of horizontal error 
signal HERR. Accordingly, the concepts of the 
present invention will be explained only in regard 
to the manipulation of the error signal HERR. 

According to a first embodiment of the present 
invention the user/operator of the tracking system 
selects the desired style or mode in which to track 
a subject. This selection is done upon powering the 
system, as indicated by block 78 of FIG. 5. The 
user communicates the selected mode to digital 
processor 46 with the two-bit STYLE signal, supplied 
over line 60. Optionally, processor 46 may be 
programmed to select a default set of parameters 
representing the initial tracking mode or style upon 
power up. Upon the selection of the desired mode, 
processor 46 sets all relevant variables to the 
appropriate values associated with the particular 
style of tracking selected. 

The process of measuring the horizontal and 
vertical angular errors between the position of the 
tracked subject and the direction in which the 
tracking device/earner a is pointing, as represented 
by block 80 of FIG. 5, is described above and 
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entails those functions to the point where error 
signals VERR and HERR are presented to digital 
processor 46. Likewise, the conversion of 4-bit 
offset binary signals and into DC voltages to 
control motors 42 and 44, respectively, as 
represented by block 90 of FIG. 5, is described in 
detail in U.S. Patent Application Serial No. 
07/396,987, now allowed, and will not be described 
further. The algorithmic steps represented by 
blocks 82, 84, 86 and 88 are performed by digital 
processor 46 in conjunction with the STYLE signal 
and are described in detail below. 

Deadzone Controller 

A first process as executed by digital processor 
46 to produce a tracking style which is more 
pleasing to human viewers includes a deadzone 
controller, represented by process block 82 in Fig. 
5 and implemented with the algorithmic steps 
illustrated in Fig. 6A. Unlike existing tracking 
systems where the desire to keep the subject 
centered at all times within the camera frame takes 
precedence over other esthetic goals, the present 
invention allows the subject to move anywhere within 
a central portion of the camera frame referred to as 
the "deadzone", without adjusting the camera 
position. 

The deadzone controller is implemented with a 
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number of variables, stored in digital processor 46, 
which represent various system parameters. The 
value of these parameters varies according to the 
desired tracking style specified by the user. 
Processor 46 manipulates the variables to achieve 
the desired controller function. 

The deadzone is described by a pair of angular 
values (DEAD LEFT, DEAD RIGHT) that correspond to 
the range of angular error that is tolerated in each 
direction before the camera position is readjusted. 
Initially, the deadzone is symmetric about the 
center of the subject and is set to (- THETA , 
+ THETA), where the variable THETA is an angular 
value and is a parameter of the tracking context. 
The total angular range of the deadzone is equal to 
twice the value of THETA. 

The ERROR variable represents the value of the 
angular ERROR along one axis between the tracked 
subject and the direction of the tracking 
device/camera. The deadzone controller takes the 
value of the ERROR variable and remaps the variable 
into a new ERROR variable which accounts for the 
deadzone. For the purposes of this exemplary 
embodiment, the ERROR variable will represent the 
angular error on the horizontal axis HERR, however 
it could eguivalently represent the angular error 
along the vertical axis, VERR. 
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Referring to FIG. 6A, the deadzone is 
represented by the values (DEAD LEFT, DEAD RIGHT). 
Digital processor 4 6 then determines whether the 
value of ERROR is still within the deadzone, as 
indicated by decisional step 96. If the value of 
the ERROR variable is within the deadzone, the ERROR 
variable is set to zero, as indicated by processing 
step 94. When the value of ERROR variable is not 
within the deadzone, the subject has moved out of 
either side of the deadzone. 

Once it is determined that the value of ERROR 
is not within the deadzone, as determined by step 
96, the direction of the ERROR is determined, as 
indicated by decisional step 98. If the value of 
ERROR is to the left of the deadzone, the value of 
DEAD LEFT is subtracted from the value of ERROR, as 
indicated by processing step 100. Conversely, if 
the value of ERROR is to the right of the deadzone, 
the value of DEAD RIGHT is subtracted from the value 
of ERROR, as indicated by processing step 102. In 
this manner, as the subject moves out of either side 
of the deadzone the angular error is transformed by 
subtracting the edge of the deadzone so that it 
describes how far in degrees from the edge of the 
deadzone the subject is rather than how far from the 
center of the camera frame. 

Ultimately, the deadzone may be implemented as a 
zone of reduced response. In this implementation 
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the response to the system, i.e. the relation 
between ERROR and MOTOR SPEED , in the reduced 
response zone can be set to a very low gain rather 
than a gain of zero. This implementation allows the 
system to slowly recenter the camera on the tracked 
subject instead of not moving at all as in the case 
with the deadzone implementation. However, such an 
implementation requires very close control over the 
motors 42 and 44 at low speeds, making the 
implementation practical only with precision 
stepping motors and not typical analog motor 
controllers. In addition, such an implementation 
complicates the processing performed by the 
directional continuity controller and motion 
controller, as explained hereinafter. 

Deadzone Shift Controller 

A second process executed by digital processor 
46 comprises a deadzone shift controller, 
represented by block 84 in Fig. 5 and implemented by 
the algorithmic steps illustrated in Fig. 6B. As a 
subject moves out of the deadzone and the camera 
begins to track the subject, the deadzone shifts. 
The purpose of the deadzone shift is to allow the 
camera to catch-up with the moving subject rather 
than always lagging by THETA. Before the deadzone 
can be shifted, a number of- limits are initialized. 
The variable MAXDEAD represents the value of THETA 
plus the maximum number of degrees that one edge of 
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the deadzone may be shifted toward the center of the 
camera frame. The MAXDEAD value is a function of 
motor speed. The motor speed for which MAXDEAD is 
greater than THETA, its minimum value, is the point 
at which the deadzone is allowed to shift. The 
motor speed for which MAXDEAD is equal to (2 • 
THETA) is the point at which the deadzone has 
shifted sufficiently for the tracking system to 
begin leading the subject. 

The variable DEADSHIFT represents the maximum 
rate in degrees per second that the deadzone may be 
shifted, and is likewise a function of the motor 
speed. The value of DEADSHIFT is typically some 
small fraction of the motor speed. 

Before shifting the deadzone, the values of the 
variables of MAXDEAD and DEADSHIFT are set to a 
function of the value of MOTOR SPEED , which 
represents the current value of the speed of the 
motor 44, as indicated by processing step 104. 

The shift of the deadzone occurs in the 
following manner.- The value of ERROR is compared to 
the right edge of the deadzone, DEAD RIGHT, as 
indicated by decisional step 106. If the value of 
ERROR is beyond the right edge of the deadzone, the 
deadzone is shifted left, as indicated by processing 
step 108. This leftward shifting is accomplished by 
subtracting the value of DEADSHIFT from both the 
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value of DEAD LEFT and the value of DEAD RIGHT. If 
the value of ERROR is not beyond the right edge of 
the deadzone, a comparison is made to determine 
whether the ERROR value is beyond the left edge of 
the deadzone. as indicated by decisional step 110. 
If the value of ERROR is beyond the left edge of the 
deadzone, the deadzone is shifted to the right, as 
indicated by processing step 112. This rightward 
shifting of the deadzone is accomplished by adding 
the value of DEADSHIFT to both the value of DEAD 
RIGHT and the value of DEAD LEFT. 

The new values of DEAD LEFT and DEAD RIGHT now 
represent the shifted deadzone which is then 
compared to the value MAXDEAD to determine if the 
deadzone has been shifted beyond. the allowable limit 
of MAXDEAD. To accomplish this, the value of DEAD 
LEFT is compared to the value of -MAXDEAD, as 
indicated by decisional step 114. If the left edge 
of the deadzone is beyond the allowable limit, i.e. 
DEAD LEFT < —MAXDEAD , the deadzone is shifted to the 
left limit. This is accomplished by setting DEAD 
LEFT to -MAXDEAD and setting DEAD RIGHT equal to 
(2 • THETA - MAXDEAD), as indicated by processing 
step 116. 

If the left edge of the deadzone is within the 
allowable limit, the right edge of the deadzone, 
DEAD RIGHT, is compared to the value of MAXDEAD, as 
indicated by the decisional step 118. If the right 
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edge of the deadzone is beyond the allowable limit, 
the deadzone is shifted to the right limit by 
setting DEAD RIGHT equal to MAXDEAD and setting DEAD 
LEFT equal to (MAXDEAD - 2 • THETA) , as indicated 
by processing step 120. 

To exemplify the above process, assume that the 
subject moves out of the right side of the deadzone 
and the camera begins to track the subject. Since 
the camera is in motion the value of MAXDEAD will be 
greater that THETA, its minimal centered value, 
allowing the deadzone to shift in the direction 
opposite to the direction of the camera's motion. 
If THETA is given a value of 5°, the initial 
deadzone is (-5, +5) degrees, with a current motor 
speed of 25° per second, typical values could be 9° 
for the maximum allowable deadzone value, MAXDEAD, 
and 1° per second for the maximum rate of deadzone 
shift, DEADSHIFT. Over a period of 4 seconds, the 
deadzone will shift from (-5, +5) to (-9, +1). At 
this point, the camera will be trailing the subject 
by 1- rather than the 5- that would result if the 
deadzone had not been shifted. 

As the camera moves at a faster speed the limit 
of the deadzone, MAXDEAD, will eventually shift to 
greater than 10° and the deadzone will begin to 
shift to (< -10, <0), with the effect of leading the 
tracked subject. As the subject slows down, the 
maximum deadzone, MAXDEAD, will decrease again to 
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5°, and the deadzone will begin to shift back 
towards its normal rest value of (-5, +5) which 
corresponds to its normal centered position, 
allowing the subject to move within the symmetric 
deadzone in the camera frame. 

The size of the deadzone, (2 • THETA) ; the 
function controlling the rate at which the deadzone 
is shifted, DEADSHIFT; and the function controlling 
the amount the moving subject is led by the camera, 
MAXDEAD, are all parameters that may be adjusted 
depending upon the tracking strategy, as specified 
by the user. Note that the choice of the DEADSHIFT 
function effects the overall stability of the 
control servo-loop of the system and should be 
chosen to avoid instability, as can be appreciated 
by one skilled in the art. 

Continuity Control 

t 

A third process executed by digital processor 46 
to produce a tracking style which is more pleasing 
to human viewers includes a directional continuity 
controller, represented by process block 86 in Fig. 
5 and implemented by the algorithmic steps 
illustrated in Fig. 6C. Human viewers are very 
sensitive to changes in the direction of the motion 
of an image. Accordingly, it is important to 
minimize the number of times the direction of camera 
motion changes. Viewers are particularly sensitive 
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to directional changes that occur either at low 
speeds or when the tracking device/camera moves from 
high speed motion to a stop and reverses direction. 
The directional continuity controller guarantees 
that there is not a series of directional changes 
that would be annoying to the viewer. Like the dead 
zone and dead zone shift controllers, the 
directional continuity controller is implemented 
with a number of variables, stored in digital 
processor 46 and representing various system 
parameters. The values of these parameters vary 
according to the desired tracking modes specified by 
the user. Processor 46 manipulates the variables to 
achieve the desired controller function. 

The variables relevant to the directional 
continuity controller are as follows. The LAST 
DIRECT variable represents, the sign of the 
direction in which the camera was last moving. The 
ERROR DIRECT represents, the sign of the angular 
error between the camera direction and the subjects 
position, i.e., the direction the camera must move 
to close-in on the subject. The STABLE TIME 
variable represents, in seconds, the elapsed time 
since the last motion of the camera. The REVERSE 
DELAY variable represents the minimum time that the 
subject is required to move in a new direction 
before the camera will begin to move. Until the 
time specified by the REVERSE DELAY variable is met 
the camera is not allowed to change direction. Note 
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that some portion of the REVERSE DELAY time will be 
spent as the subject crosses the dead zone of the 
camera frame. 

The algorithmic steps executed by the 
directional continuity controller are illustrated in 
Fig. 6C. At any point in time, the LAST DIRECT and 
STABLE TIME variables represent the last direction 
of camera motion and the elapsed time since such 
motion, respectively. If other control processes 
try to move the camera in the same direction, such 
motion is allowed and the elapsed time is reset. To 
accomplish this, processor 46 compares the value of 
the LAST DIRECT variable to zero, as indicated by 
decisional step 122. If the value of LAST DIRECT 
equals zero, indicating that the camera has not 
recently been in motion, the camera is allowed to 
move in the direction of the error, as indicated by 
processing step 126. During step 126 the value of 
the STABLE TIME variable is set to zero and the 
value of the LAST DIRECT variable is set to the 
value of the ERROR DIRECT variable. 

If in decisional step 122, the LAST DIRECT 
variable is not equal to zero, indicating that the 
camera has recently been in motion, the value of the 
ERROR DIRECT variable is compared to the LAST DIRECT 
variable, as indicated by decisional step 126. If 
the ERROR DIRECT variable equals the LAST DIRECT 
variable, indicating that the error direction and 
the last camera direction are the same, processing 
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step 126 is executed as previously described. 

If the error direction and the last camera 
direction are not equal, the elapsed time since the 
last motion, STABLE TIME , is compared to the REVERSE 
DELAY constant, as indicated by procedural step 
128. If the value of the STABLE TIME variable is 
greater than or equal to the REVERSE DELAY value, 
then processing step 126 is executed as previously 
described. If however, the elapsed time since the 
last motion is less than the time specified in the 
REVERSE DELAY variable the camera is not allowed to 
reverse directions, as indicated by processing step 
130. In processing step 130 the value of the STABLE 
TIME is incremented and the ERROR variable is set to 
zero. The value of the LAST DIRECT variable remains 
unchanged . 

In this manner, the directional continuity 
controller allows the camera to move in the 
direction of its most recent motion. However, a 
minimum threshhold of time is required before the 
camera is allowed to reverse directions. The value 
of the time delay specified in the REVERSE DELAY 
variable is a parameter of the tracking context, as 
indicated by the user-specified STYLE signal. 
Typical ranges for the value of the REVERSE DELAY 
variable can be from 1/6 of a second to greater than 
5 seconds, without causing instability in the servo 
loop of the tracking system. 
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Motion Controller 

A fourth process executed by digital processor 
46 includes a motion controller, represented by 
process block 88 in Fig. 5 and implemented by the 
algorithmic steps illustrated in Fig. 6D. 

In most video pictures, the background typically 
represents a greater portion of the picture than 
does the subject. If the motion of the scene 
against the background is jerky, the picture will be 
disconserting to the viewer, even if this results 
from tracking the inconsistent motions of the 
subject. The motion controller performs a pseudo 
filtering function in the response of the system by 
selecting acceleration time constants that are 
different from the deceleration time constants for 
the motors driving the camera. As a result, the 
camera motion starts smoothly while avoiding any 
tendency to overshoot the subject when the motion is 
halted. In addition to the filtering function, the 
motion controller places limits on the maximum 
velocity and acceleration of the camera depending on 
the context of the subject "s motion. As with the 
other process controllers, the motion controller is 
implemented with a number of variables, stored in 
digital processor 46 and representing various system 
parameters. The value of these parameters varies 
according to the desired tracking modes selected by 
the user . 
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The variables relevant to the motion controller 
are as follows. The variable ERROR is defined as 
similar to that used by the dead zone controller. 
The variable LAS TERROR represents the previous value 
of ERROR . The variable MAXACC represents the 
maximum allowable acceleration, measured in degrees 
per second squared, for the selected tracking mode. 
The variable KAXVEL represents the maximum allowable 
velocity, measure in degree per second, for the 
selected tracking mode. The variable K ACC 
represents the exponential smoothing time constant 
that controls the rate of which the motor 4 4 is 
accelerated. Tne value (1 - K ACC) is the fraction 
of the exponentially averaged history of the signal 
that is left after one time step. The variable K 
DEACC represents the exponential smoothing time 
constant that controls the speed at which motor 44 
is decelerated. The value < 1 — K DEACC) is the 
fraction of the exponentially averaged history of 
the signal that is left after one time step. 
Typically the variable K DEACC is greater than or 
equal to the variable K ACC to make the acceleration 
as smooth as possible while avoiding overshooting of 
the subject when the camera motion stops. The 
values of both K ACC and K DEACC are in the range 
between zero and one. 

The motion controller operates as follows. If 
the value of ERROR would lead to a camera velocity 
that is greater than the maximum allowable velocity 
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prescribed by the tracking mode, the velocity is 
limited. This is accomplished by comparing the 
value of ERROR to the value of MAXVEL, as indicated 
by decisional step 132. If the value of ERROR is 
greater than value of MAXVEL , the velocity is 
limited by setting ERROR equal to the value of 
MAXVEL, as indicated by procedural step 134. If the 
value of ERROR is not greater than MAXVEL in step 
132 it is compared to the value of -MAXVEL, as 
indicated by decisional step 136. If the value of 
error is less than -MAXVEL, the velocity is again 
limited by setting ERROR equal to -MAXVEL, as 
indicated by process step 138 . 

If the difference between the current value of 
ERROR and the previous value of ERROR, LAST ERROR 
(which is the acceleration) is greater than the 
maximum acceleration prescribed by the tracking 
context, the camera velocity is limited to the 
current camera velocity plus the maximum 
acceleration. To accomplish this, the value of 
ERROR is compared to the sum of the LAST ERROR and 
MAXACC, as indicated by decisional step 140. If the 
value of ERROR is greater in step 140, the 
acceleration is limited by setting the value of 
ERROR to the sum of LAST ERROR and MAXACC, as 
indicated by process step 142. otherwise, the value 
of ERROR is compared to the current velocity minus 
the maximum allowable acceleration, as indicated by 
decisional step 144. If the value of ERROR is less 
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than the value of LAST ERROR minus MAXACC, the 
acceleration is again limited by setting the value 
of ERROR to LAST ERROR minus MAXACC , as indicated by 
processing step 146. 

After any acceleration or velocity limiting, 
digital processor 46 determines whether the camera 
is accelerating, as indicated by decisional step 
148. If the camera is accelerating, a smoother 
acceleration is achieved as indicated by process 
step 150. A smooth acceleration is achieved by 
computing a smooth error value, NSERROR using the 
value of ERROR and the value of the last smoothed 
error, LSERROR , with Eq. (D below. 

NSERROR = (1-K ACC) • LSERROR + K ACC • ERROR 
Eq. (1) 

Process step 150 is the equivalent of low pass 
filtering. The new smooth error, NSERROR. of Eq. (1) 
is then used to control the speed of motor 44. 

If the camera is not accelerating, processor 46 
determines if the camera is decelerating, as indicated 
by decisional step 152. If the camera is decelerating, 
a smooth decelaration is performed as indicated by the 
processing step 154. The smooth deceleration is 
achieved by computing a smooth error value, NSERROR 
using the value of ERROR and the value of the last 
smoothed error, LSERROR with Eq. (2) below. 
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NSERROR = (1-K DEACC ) • LSERROR + K DEACC • ERROR 
Eq . ( 2 ) 

The above pseudo low pass filtering technique uses 
different time constants when accelerating and 
decelarating. The sensitivity of viewers to the 
smoothness of acceleration is greater than the 
sensitivity to the smoothness of decelar at ion . In a 
preferred embodiment the ratio between K ACC and K 
DEACC is typically 1:2. 

In the first embodiment of the present invention 
the user of the tracking system specifies a tracking 
context so that the system performance is specifically 
tuned for the context of the subject's motion. A 
typical selection of modes or tracking styles may 
include slow, general and fast action modes. A slow 
action mode, good for lectures, walking sports, playing 
with young children or any activity without running, is 
be characterized by slow, smooth motion with a tendency 
to lag the subject if the subject engages in a quick 
motion. A general action mode, good for general play 
is appropriate for smooth motion with a tendency to lag 
the subject only when the subject is at a full run. A 
fast action mode, good for tennis, basketball, soccer 
or any activity with full speed running and quick 
changes of motion would be appropriate for quickly 
tracking even the fastest moving subject and has a 
tendency to react more quickly than is visually 
pleasing if the action isn't fast enough to mask it. 
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Automatic Contexts Determination Controller 

According to the second embodiment of the 
present invention, the user-supplied STYLE signal, 
describing the desired tracking style, is replaced 
with a signal generated by digital processor 46, 
based on the past history of the tracked subject's 
motion as described by the most recent values for 
some of the system variables, as explained 
hereinafter . 

Figure 7 illustrates processes executed by 
digital processor 46 to process signals VERR and 
HERE. . In this embodiment, processor 46 
automatically adjusts the previously described 
processes so that the response characteristics of 
the tracking system give it attributes that result 
in videos that are pleasing to the human viewer, 
Referring to Figure 7, the algorithmic process 
represented by blocks 78, 80, 82, 84, 86, 88, and 90 
are similar to those described with reference to 
Figure 5 and the first embodiment of the present 
invention . 

A fifth process executed by digital processor 46 
includes an automatic context controller, 
represented by process block 92 in Figure 7 and 
implemented by the algorithmic steps illustrated in 
Figure 8. 
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Since a part of the function of the tracking 
system is to determine the position of the subject 
being tracked, there is a significant amount of 
information available that can be used to predict 
the style of future motions. Typically, the real 
activities that the subject engages in are quite 
stable in their range of motions. In the second 
embodiment of the present invention the parameters 
of the subject's motion are stored and then compared 
with the parameters expected to be associated with 
such a tracking context. The extracted parameters 
include: the spectral characteristics of the motion, 
the percentage of time the current context tracks 
properly, the maximum angular acceleration of the 
subject, and the minimum angular speed of the 
subject. These parameters are chosen more for ease 
of calculation than as being the best parameter. 

The variables relevant to the automatic context 
determination controller are as follows. The 
CONTEXT variable is the equivalent to the STYLE 
signal and designates the currently selected 
tracking context or mode. The MISS TRACK I NG variable 
represents the length of time the subject is outside 
of the dead zone. The MAXMISSTRACK variable 
represents the maximum allowable percentage of time 
the subject may be allowed outside the dead zone for 
the particular tracking style selected by the 
CONTEXT variable. The AVG MOVE variable represents 
the average time the subject moved in a single 
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direction. The MOTOR SPEED variable is the same as 
previously defined regarding the first embodiment. 
The DELTA ERROR variable represents the change in 
angular error, in degrees, since the angular error 
of the subject was last measured. DELTA ERROR is 
added to the MOTOR SPEED to give the speed of the 
subject, CURRENT SPEED. The DELTA SPEED variable 
represents the change in CURRENT SPEED, in degrees 
per second, since the angular error of the subject 
was last measured. The MAX SPEED variable 
represents the maximum angular speed of the 
subject. The MAX ACC variable represents the 
maximum angular acceleration of the subject. 

Following processing by motion controller 88, 
the automatic context determination controller 92 
determines whether the current tracking context is 
appropriate for the motion of the subject. 
Referring to Figure 8, with each new value of ERROR, 
the amount of time in which the current system mode 
mistracked the subject is computed, as indicated in 
processing step 160. The absolute value of the 
ERROR variable is used to determine whether the 
subject is currently being mistracked. If the 
absolute value of the ERROR variable is less than or 
equal to a maximum allowable error, the value of 
MI SSTRACKING is not incremented, indicating that the 
subject is still moving within the acceptable zone. 
However, if the absolute value of the ERROR variable 
is greater than the maximum allowable angular error, 
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the value of MISS TRACK I NG is incremented, indicating 
that the subject has left the acceptable zone. 

The percentage of the time the system is 
mistracking is calculated by dividing MISSTRACKING 
by the total time, and compared to the maximum 
allowable percentage for the current mode, 
MAXMISSTRACK, as indicated in decisional step 162. 
If the percent of time the system is mistracking the 
value of MISSTRACK is greater than the value of 
MAXMISSTRACK, the value of the CONTEXT variable is 
set to the next faster context or tracking mode, as 
indicated by processing step 164. Next the maximum 
angular speed of the subject, MAX SPEED, is 
calculated as indicated by processing step 166. 
This is done by setting the value of MAX SPEED to 
the maximum of the last value of MAX SPEED and the 
absolute value of the current speed of the subject, 
CURRENT SPEED, which is computed as in Eg. (3) below. 

CURRENT SPEED - (MOTOR SPEED + DELTA ERROR) Eq . (3) 

The absolute value of the speed of the subject 
is used so the value of MAX SPEED will be direction 
independent . 

Next, the maximum angular acceleration of the 
subject, MAX ACC, is calculated as indicated by 
processing block 168. This is done by setting the 
value of MAX ACC to the maximum of the last value of 
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MAX ACC and the absolute value of the current 
acceleration of the subject, CURRENT ACC, which is 
computed as in Eq. 4 below: 

CURRENT ACC = DELTA SPEED Eq . (4) 

The absolute value of the difference in speed of 
the subject is used so the value of MAX ACC will be 
direction independent. 

Finally, the average time of the subject's 
movement is computed as indicated by processing step 
170. The average time is calculated by simply 
computing the average time between directional 
changes and setting the AVG MOVE variable to such 
average . 

Associated with each available context or mode 
is a range of values that correspond to appropriate 
ranges of motion of the subject. A determination is 
made whether the current values characterizing the 
subject's motion more closely match those of a 
slower context, as indicated by decisional step 
172. If the characteristics of the subject's motion 
will closely match a slower mode or context, the 
value of CONTEXT is set to the next slower context, 
as indicated by processing step 174. Otherwise, the 
tracking context remains the same. 
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Variable Ranges 

Depending on the actual physical implementation 
of the tracking system, various other variables may 
be relevant. For instance, a GAIN variable may be 
necessary for controlling the overall gain of the 
servo-loop in the tracking system. A MOTOR OFFSET 
value would allow the system to account for the 
minimum start up voltages required when utilizing 
analog motors. By adding a MOTOR OFFSET to the 
ERROR value during the motor control process the 
zero-crossing response of such a motor can be evened 
out. Setting the MOTOR OFFSET variable to zero has 
the effect of adding to the dead zone in a way which 
is difficult to compensate for with other variables 
in the system. 

Since the entire tracking system disclosed in either 
embodiment of the present invention forms a single 
servo system, it is difficult to give actual values 
for the variables contained in each different mode 
parameter set. Such parameter values are heavily 
dependent on the specific considerations of the 
entire electro-mechanical system. However, below 
are listed a number of ratios which represent a 
reasonable range of values for select variables 
between a slow action tracking mode and a fast 
action tracking mode. The overall gain of the 
tracking system, i.e., the response of MOTOR SPEED 
to ERROR, may range over a factor of 4:1. The ratio 
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of K ACC to K DEACC may range over a factor of 1:3. 
The values of K ACC and K DEACC may each range over 
a factor of 3:1. The REVERSE DELAY time may range 
from 1/6 second to 6 seconds. Finally, the value of 
THETA may range over a factor of 5:1. 

Alternate Embodiments 

Although the present invention has been 
described with reference to a sampled, ultrasonic 
tracking system which uses DC motors, it will be 
obvious to one skilled in the art that the present 
invention is equally applicable to any tracking 
system. 

Such systems may be active, as in the exemplary 
embodiments, or may be passive tracking systems, 
such as infrared tracking systems which do not 
require a transmitter to track a subject. Other 
such tracking systems include systems which use 
frame to frame motion comparison and pattern 
recognition techniques to identify portions of an 
image that are to be maintained within the camera 
frame. Still other systems may be sampled, as in 
the exemplary embodiments, or may be continuous, and 
may use infrared, radio frequency, or sonic 
transmission to communicate between the 
transmitter/receiver. Other systems may use 
precision stepping motors to control the camera or 
tracking device instead of analog motor controllers. 
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In the above exemplary embodiments, the 
algorithmic processes represented by blocks 78, 80, 
82, 84, 86, 88, 90 and 92 are implemented with a 
plurality of system variables and a programmed 
digital processer. It will be obvious to those 
reasonably skilled in the art that functionally 
equivalent processes may be achieved with an 
implementation using small, medium or large scale 
logic components as well as analog circuit 
elements. Such hardware implementations are well 
within the scope of those reasonably skilled in the 
art and are within the scope and spirit of the 
present invention. 

The above invention has been described with 
reference to the horizontal angular error signal, 
HERR, which represents one axis of a multiple axis 
system. It is obvious to those skilled in the art 
how the control functions disclosed herein are 
equally applicable to the vertical angular error 
signal, VERR, provided the designer accounts for the 
differing sensitivity of the human eye to vertical 
axis motion. It will be further obvious to those 
reasonably skilled in the art that digital processor 
46 may perform the control functions for both the 
horizontal and the vertical angular error signals, 
as shown in FIG. 3, or that two such digital 
processors may be used, one for each axis of motion. 
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Further, since many activities have different 
motions in each axis, contemplated tracking systems 
may allow different tracking modes to be assigned to 
each axis. Conversely, a contemplated tracking 
system may provide for automatically or 
user-selected tracking modes which fix the 
horizontal and vertical axis tracking styles to each 
other, the parameters of such modes being defined 
for a specific two-axis activity. 

It should be further understood that the 
concepts of the present invention apply to any 
tracking system, which given a set of criteria, uses 
the particular parameters of the tracking system to 
optimize the system performance to- satisfy the 
criteria. The exemplary embodiment listed a nuinber 
of visual criteria. It will be obvious to those 
reasonably skilled in the art that other criteria 
associated with the smart tracking of a subject, but 
not disclosed herein may be similarly achieved with 
the apparatus and methods disclosed in the present 
invention, given the proper system parameters. Such 
criteria and the means for optimizing the tracking 
system performance to meet the criteria are within 
the scope of the present invention. 

Accordingly, it will be appreciated that the 
detailed disclosure has been presented by way of 
example only and is not intended to be limiting. 
Various alternative, modifications and improvements 



BNSDOCID: <WO 9204641 A 1_IA> 



WO 92/04641 PCT/US9 1/063 16 

- 39 - 



will readily occur to those skilled in the art, and 
may be practiced without departing from the spirit 
and scope of the invention. The invention is 
limited only as required by the following claims and 
equivalents thereto. 

What is claimed is: 
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1. A system for permitting a device to 
point at and track the movements of a moving 
subject , comprising : 

a. a signal source for generating signals 
which can be tracked; 

b. control means for receiving signals 
from the source and for generating relative 
orientation data therefrom in the form of control 
signals which vary as continuous analog functions 
with changes in relative orientation; 

c. means for storing plural sets of data 
values, each of said data value sets representing 
Selected values for processing said control signals 
in accordance with the anticipated motion of the 
subject; 

d. means for selecting one of said sets of 
data values; 

e. means for processing said control 
signals in accordance with the one of said sets of 
data values selected by said selecting means to 
produce processed control signals; and 

f . a drive unit for moving the device about 
at least one axis in response to said processed 
control signals to maintain the device pointed at the 
subject . 

2. The system of Claim 1 wherein each of said 
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data value sets comprises data values representing 
selected acceleration rates of the device and 
selected deceleration rates of the device, for a 
particular range of anticipated motion by the subject. 

3 . The system of Claim 1 where each of said data 
value sets comprises data values representing a 
selected angular velocity of the device, and a maximum 
allowable angular error between the axis of the device 
and the subject, for a particular range of anticipated 
motion by the subject. 

4. The system of any of Claims 1, 2 or 3 wherein 
said means for selecting comprises a user-defined 
signal for indicating which of said plural sets of 
data values is to be selected. 

5. The system of any of Claims 1, 2, or 3 wherein 
said means for selecting comprises means for 
automatically selecting one of said plural sets of 
data values in anticipation of future relative motion 
of the subject based on past motion of the subject. 

6. A method of intelligently and automatically 
tracking a moving subject with a camera comprising the 
steps of: 

a. determining a last direction in which the 
camera moved about an axis; 

b. calculating an elapsed time value representing 
the elapsed time interval since the camera last moved; 
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c. determining an error direction representing 
the present orientation of the subject relative to the 
earner a ; 

d. comparing said last direction with said error 
direction; 

e. moving the camera about the axis in the error 
direction if said last direction and said error 
direction are equal; 

f. comparing said elapsed time value with a 
predetermined value if said last direction and said 
error direction are not equal; and 

g. moving the camera about the axis in the error 
direction if the elapsed time value at least equals 
the predetermined value. 

7. The method of claim 6 further comprising the 
step of : 

h. maintaining the current angular orientation 
of the camera if the elapsed time value is less than 
the predetermined value. 

8. The method of claim 6 wherein said 
predetermined value is selected to represent the 
average time between directional changes by a subject 
for a predetermined class of motions. 

9. An apparatus for intelligently and 
automatically tracking a moving subject with a camera 
comprising : 

a. means for determining an error direction 
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representing the present orientation of the subject 
relative to the camera; 

b. means for sensing a last direction in which 
the camera moved about an axis; 

c. means for calculating an elapsed time value 
representing the elapsed time interval since the 
camera last moved; 

d. means for comparing the last direction with 
the error direction, and for comparing the elapsed 
time value with a predetermined value if the error 
direction and the last direction are not equal; and 

e. a drive unit, responsive to said means for 
determining and said means for comparing, for moving 
the camera in the error direction about said one axis 
if the elapsed time value at least equals the 
predetermined value or the last direction equals the 
error direction. 

10. The apparatus of claim 9 wherein said means 
for determining comprises: 

control means for receiving source signals from 
the subject and for continuously generating therefrom 
control signals representing the angular error 
between the position of the subject and the camera 
along at least one plane. 

11. A method of intelligently and automatically 
tracking a moving subject within the frame of a 
camera comprising the steps of: 

a. moving the camera about an axis to bring the 



BNSDOCID: <WO 9204641A1_IA> 



WO 92/04641 



PCT/US91/06316 



- 44 - 



subject within the camera frame; 

b. establishing predetermined limits for the 
angular velocity and angular acceleration of the 
camera about the axis; 

c. defining a central zone within the camera 
frame and substantially symmetrical about the center 
of the subject; 

d. moving the camera about the axis at rates not 
to exceed the predetermined limits to maintain the 
subject within the zone; 

e. calculating a first percentage representing 
the total time the subject was outside said central 
zone during a given time interval; 

f. calculating the maximum angular speed and 
maximum angular acceleration of the subject within 
said given time interval; and 

g. redefining said predetermined limits for the 
angular acceleration and angular velocity of the 
camera in accordance with the anticipated motion of 
the subject, 

12. The method of claim 11 wherein step (g) 
comprises comparing the first percentage with a 
predefined maximum allowable percentage and selecting 
new predetermined limits having increased values for 
the angular velocity and angular acceleration of the 
camera when the first percentage exceeds the 
established predetermined limits. 

13. The method of claim 11 wherein step (g) 



BNSDOCID: <WO 9204641 A 1_l A > 



WO 92/04641 



PCT/US91/063I6 



- 45 - 



comprises selecting new predetermined limits having 
decreased values for the angular velocity and angular 
acceleration of the camera when the maximum angular 
speed and maximum angular acceleration of the subject 
are less than the predetermined limits for the 
angular velocity and angular acceleration of the 
camera . 

14. An apparatus for permitting a camera to 
maintain a moving subject within the frame of a 
camera in a manner which is visually pleasing to a 
viewer, the apparatus comprising: 

a. means for storing a plurality of 
predetermined limits for the angular velocity and 
angular acceleration of the camera about an axis; 

b. a drive unit, responsive to said means for 
storing, for moving the camera about the axis at 
rates not in excess of the predetermined limits to 
maintain the subject within the zone; 

c. means for defining a central zone within the 
camera frame and substantially symmetrical about the 
center of the subject; 

d. first means for calculating a first 
percentage representing the total time the subject 
was outside the central zone during a given time 
interval ; 

e. second means for calculating the maximum 
angular velocity and maximum angular acceleration of 
the subject within the given time interval; and 

f. means, responsive to said first and said 
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second means, for selecting new predetermined limits 
from the means for storing for the angular 
acceleration and angular velocity of the camera in 
anticipation of future motion of the subject. 

15. The apparatus of claim 14 wherein the means 
for selecting comprises means for comparing the first 
percentage with a predetermined maximum allowable 
percentage stored in said means for storing and for 
selecting new predetermined limits for the angular 
velocity and the angular acceleration of the camera 
when the first percentage exceeds the predetermined 
maximum allowable percentage. 

16. The apparatus of claim 14 wherein said means 
for selecting comprises means for comparing the 
maximum angular velocity and maximum angular 
acceleration of the subject with the current 
predetermined limits for angular velocity and angular 
acceleration of the camera, and for selecting new 
predetermined limits for the angular acceleration and 
angular velocity of the camera when the maximum 
angular velocity and maximum angular acceleration of 
the subject are substantially less than or 
substantially greater than the current predetermined 
limits . 

17. A method of intelligently and automatically 
tracking a moving subject within the frame of a 
camera comprising the steps of: 
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a. defining a central zone within the camera 
frame by a pair of angular coordinates along one axis 
and substantially symmetric about the center of the 
subject at a first time; 

b. determining an angular error along said one 
axis between the subject and the camera at a second 
time; 

c. determining whether the angular error is 
within the central zone; 

d. determining an error direction representing 
the direction the camera must move to reduce the 
angular error at the second time, if the angular 
error is not within the central zone; 

e. moving the camera about said one axis in the 
error direction; and 

f. redefining the angular coordinates of the 
central zone so that the central zone is shifted in a 
direction which is opposite the error direction. 

18. The method of claim 17 wherein step (f) 
comprises the steps of: 

f.l subtracting the value of one of the angular 
coordinates of the central zone from the value of the 
angular error to form a difference value; and 

f.2 subtracting the difference value from each 
of the angular coordinates to form new angular 
coordinates . 

19. The method of claim 17 further comprising 
the step of: 
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g. maintaining, the camera in its current 
orientation if the angular error is within the 
central zone. 

20. An apparatus for intelligently and 
automatically tracking a moving subject within the 
frame of a camera comprising: 

a. means for defining a central zone within the 
camera frame by a pair of angular coordinates along 
one axis and substantially symmetric about the center 
of the subject at a first time; 

b. means for determining an angular error along 
said one axis between the subject and the camera at a 
second time; 

c. means for determining whether the angular 
error is within the central zone; 

d. means for determining an error direction 
representing the direction in which the camera must 
move to reduce the angular error at the second time, 
if the angular error is not within the central zone; 

e. a drive unit for moving the camera about said 
one axis in the error direction; and 

f . means for redefining the angular coordinates 
of the central zone so that the central zone is 
shifted in a direction which is opposite the error 
direction. 

21. The apparatus of claim 20 wherein said means 
for redefining comprises: 

means for subtracting the value of one of the 
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angular coordinates defining the central zone from 
the value of the angular error to form a difference 
value; and 

means for subtracting the difference value from 
each of the angular coordinates to form new angular 
coordinates . 

22. An apparatus for permitting a camera to 
maintain a moving subject within the camera frame in 
a manner that is visually pleasing to a viewer, the 
system comprising: 

a. a subject for generating signals which can be 

tracked; 

b. control means for receiving signals from the 
subject and for continuously generating relative 
orientation data therefrom in the form of control 
signals, said control signals representing the 
angular error between the position of the subject and 
the camera along at least one plane; 

c. means for anticipating future relative motion 
of the subject based on past motion of the subject; 

d. means, responsive to said control signals, 
for determining whether the angular error represented 
by the control signals is within an acceptable range 
given anticipated motion by the subject, and for 
processing said control signals if said angular error 
is not within the acceptable range; and 

e. a drive unit, responsive to said processed 
control signals, for selectively moving the camera 
about an axis to maintain the subject within the 
camera frame. 



BNSDOCID: <WO 9204641A1_IA> 



WO 92/04641 



1/S 



PCT/US91/063J6 

J** 





BNSDOCID: <WO 9204641 A 1 J A> 



WO 92/04641 



PCT7US91/063I6 



2/S 








fO 








cr 


cr 


cr 


tr 


LU 


UJ 


> 





ro 
C9 




< 

ro 



< 

o 

CM 



cr 




cr 




cr 


o 




o 




O 










H 




CD 




o- 




LU 


o- 


UJ 


ro 






K) 






DE 


Ui 

o 




UJ 

o 



CD 
O 
CVJ 



CO 



SUBSTITUTE SHEET 



BNSDOCID: <WO 9204641 A 1_IA> 



WO 92/04641 



PCT/US91/06316, 



3/9 



78 



80 



POWER - UP 
MODE SELECT 



I 



MEASURE 



82 



DEADZONE 



84 



I 



SHIFT 
DEADZONE 



86 



I 



CONTINUITY 
CONTROL 



88 



I 



MOTION 
CONTROL 



90 



MOVE 



FIG. 5 



BNSDOCID: <WO 9204641 A1_IA> 



SUBSTITUTF cum- 



WO 92/04641 



PCT/US91/06316 




1 — 


MEASURE J 








MOVE J" ' 




L_ j .1 



F I G. 4 PRIOR ART 




V 



FIG.6A 



BNSDOCID: <WO 9204641A1_IA> 



SUBSTITUTE SHEET 



WO 92/04641 



104 
2_ 



© 



SET LIMITS 



5/9 



PCT/ US9 1/063 1 6, 




I 08 

) 

SHIFT 
DEADZONE 
LEFT 



112 



SHIFT 
DEADZONE 
RIGHT 



16 





> 




YES 


LI M IT 




DEAD LEFT 


YES 







120 

J 



LIMIT 
DEAD RIGHT 



FIG. 6B 



BNSDOC1D: <WO_ 



_9204641A1_IA> 



SUBSTITUTE SHEET 



WO 92/04641 



PCT/US9 1/063 16 




DIRECTION 
NOT OK 



V 



FIG.6C 



BNSDOCID: <WO 9204641 A 1JA> 



SUBSTITUTE SHEET 



WO 92/04641 



PCT/US9 1/063 16 




SUBSTITUTE SHEE1 

BNSDOCID: <WO 9204641 A 1_IA> 



WO 92/04641 



8/9 



PCT/US91/063I6 



78 



80 



POWER UP 
MODE SELECT 



I 



MEASURE 



82 



MEASURE THE ANGULAR ERROR 
BETWEEN THE CAMERA AND SUBJECT 



84 - 



DEAC 


)20NE 






SHIFT 
DE A OZONE 


1 



86 



CONTINUITY 
CONTROL 



CORRECT ERROR TO TAKE 
DEADZONE INTO CONSIDERATION 



SHIFT DEADZONE IN DIRECTION 

OPPOSITE MOTION AT RATE 
DEPENDING UPON MOTOR SPEED 



REQUIRE A PERIOD OF TIME TO 

PASS BEFORE ALLOWING 
THE ERROR TO REVERSE 



88 — 



92 



90 



MOTION 
CONTROL 



ADAPTIVELY LOW PASS FILTER ERROR 
LIMIT VELOCITY AND ACCELLERATION 



— <D 



AUTOMATIC 
CONTEXT 
CONTROL 



SELECT APPROPRIATE 
TRACKING CONTEXT 
FOR ONGOING MOTION 



— © 



MOVE 



SET MOTOR SPEED 
SO THAT MOTOR MOTION 
WILL CORRECT ERROR 



FIG. 7 



BNSDOCID: <WO 920464 1 A 1 _ I A> 



SUBSTITUTE SHEE1 



WO 92/04641 



9!9 



PCT/US91/063I6.. 



1 



160 



COMPUTE 
MISSTRACKING 




166 — 



COMPUTE 
CURRENT 
MAX SPEED 



168 



COMPUTE 
CURRENT 

MAX ACC 



170 — 



I 



COMPUTE 
AVG MOVE 




i7 



164 



SWITCH 
CONTEXT 



SWITCH 
CONTEXT 



FIG.8 



BNSDOCID: <WO 9204641 A 1 JA> 



SUBSTITUTE SurrT 



INTERNATIONAL SEARCH REPORT 

lBtcroAtio&»l Application No 



PCT/US 91/06316 




Pm PCT/IW210 <> 



BNSDOCID: <WO_ 



_9204641A1 JA> 



Iot«ro*tioo*l Application No 

m W. ,MraT* CONSIDERED TO B E RELEVANT (CONTINUED FTtOM THE SECOND SHlgh 



QabDo 



of DocDDMDt, with Urfiarioo. wb«r« oypcoyimf, of tb« wkvMl ^tnfcs 



PCT/US 91/06316 



Rji«vut to CUia No. 



US, A, 4 189 747 ( FUNARI ) 19 February 1980 

see column 1, line 20 - column 2, line 3; figure 
1 



I, 6,9, 

II, 14, 
17,20,22 



hm rCT7BAJll»«nra Ml (J i ni> l«> 



BNSDOCIO: <WO 9204641A1JA> 



ANNEX TO THE INTERNATIONAL SEARCH REPORT 

ON INTERNATIONAL PATENT APPLICATION NO. US 9106316 

SA 52605 



Tki* iiiiKX lists the aaceat fmniy 
The European Patent Office is la mo way 



rotaUmt ts tbe ittort doosmti csted 
PmhK Office EDP fite so 
for 



is Che ahave — ct 

fiveo for Che 



06/02/92 



faosJy 
«) 



US-A-4409661 



11-10-83 



None 



US-A-4064528 


20- 


•12- 


-77 


US-A- 


3840699 


08-10-74 


US-A-4189747 


19- 


-02- 


-80 


None 







ex : see Official JacreaJ mf the Eurapcan Patent Office, No. 12/82 



BNSDOCID: <WO 9204641 A 1_IA> 



WORLD INTELLECTUAL PROPERTY ORGANIZATION 
International Bureau 




PCT 

INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT) 



(51) International Patent Classification 6 : 
H04N 7/18 // G03B 15/00, 29/00 



Al 



(11) International Publication Number: WO 95/35627 

(43) International Publication Date: 28 December 1995 (28.12.95) 



(21) International Application Number: PCT/IB 95/0050 1 

(22) International Filing Date: 21 June 1995 (21.06.95) 



(30) Priority Data: 

08/263,782 



22 June 1994 (22.06.94) 



US 



(71) Applicant: PHILIPS ELECTRONICS N.V. [NUNL]; Groe- 
ncwoudseweg 1, NL-5621 BA Eindhoven (NL). 

(71) Applicant (for SE only): PHILIPS NORDEN AB [SE/SE]; 

Kottbygatan 5, Kista, S-164 85 Stockholm (SE). 

(72) Inventors: SERGEANT, Ronald, Gilbert; 3512 Brandemere 

Drive. High Point, NC 27265 (US). CORBIN, Scott, Alan; 
38 Summers Drive, Stevens, PA 17578 (US). JONES, 
Theodore, Leroy; 502 West Main Street, Akron, PA 17501 
(US). MEKROTRA, Gopi, Nath; 340 Ludwell Drive, 
Lancaster, PA 17601 (US). RANDALL, Jennifer, Lynn; 
1057 Grandview Boulevard, Lancaster, PA 17601 (US). 

(74) Agent: SCHOUTEN, Marcus, Maria; International Octrooibu- 
reau B.V., P.O. Box 220, NL-5600 AE Eindhoven (NL). 



(81) Designated States: JP, European patent (AT, BE, CH, DE, DK, 
ES, FR» GB, GR, IE. IT, LU, MC, NL, PT, SE). 



Published 

With international search report. 



(54) Title: VIDEO SURVEILLANCE SYSTEM 



20 

- — f-- r-120 



*22o \b ^22c 



U£L 



1_ 



SWnotR/MJLTlPLBER 




12a 42b 



sou 



era 



T 



j 6 



ooooooo C 
ooo 



X 



-30 




(57) Abstract 

A video surveillance system includes a plurality of remote surveillance units, each of which has a video camera mounted in a dome 
housing. The video camera is panned and/or tilted to a desired orientation within the dome by a pan motor and/or a tilt motor. The 
surveillance system also includes a keyboard for entering user commands, a joystick for controlling the positioning of the cameras, a 
central processing unit for interpreting the user commands and the joystick movements, and a communication system for issuing operating 
commands to the remote surveillance units. Each remote surveillance unit is constructed and arranged to store and to recall selected camera 
control parameters in association with selected pan and tilt coordinates in response to commands entered by the user. The camera control 
parameters include white balance, focus, and shutter speed settings for the camera. Additionally, automatic control of the camera exposure, 
white balance, and focus is provided. Video signals from the remote surveillance units are received by a video switcher/multiplexer under 
the control of a central processing unit. The video switcher/multiplexer includes an automatic phase adjustment system for synchronizing 
the video signals from the remote surveillance units. 
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Video surveillance system. 



FIELD OF THE INVENTION 

The present invention relates to video surveillance systems. More particularly, 
the invention relates to a surveillance system wherein a plurality of compact cameras are 
installed in a plurality of remote locations and wherein various operational parameters of the 
5 cameras are stored and retrieved to effect rapid adjustment of each camera. 

BACKGROUND OF THE INVENTION 

Video surveillance systems are known wherein video cameras are deployed at 
remote locations that are desired to be monitored by security personnel. In such systems, 

10 images from several cameras are displayed on one or more video monitors at a security 
console. By having a single person remotely monitoring a large number of locations, a 
considerable cost advantage can be realized compared to employing several people to be 
stationed in the locations of interest. 

Traditionally, the video cameras employed in such surveillance systems have 

15 been somewhat unsophisticated relative to video cameras used in television broadcasting 

systems or cameras intended for home use. For almost every operational feature that may be 
incorporated into a video camera, there is a corresponding parameter that must be set and/or 
adjusted by the user. The need for such adjustments is not an overwhelming problem for the 
user of a single camera. However, the use of feature-laden cameras in a surveillance system 

20 having many cameras requires a significantly greater amount of the user's attention in order 
to set or maintain adjustments of the multitude of parameters for each camera. Even if each 
camera were to be initially adjusted according to a set of pre-determined parameters, the 
required adjustments are likely to change in response to changing camera orientations, 
changing lighting conditions, or other events within a monitored area. Thus, the user of a 

25 surveillance system would be distracted from the task of providing vigilant attention to 

activities in the monitored areas. For example, simply maintaining and adjusting a plurality 
of remote video cameras in focus could present a significant distraction to the user. 

It would be desirable to provide a multiple-camera surveillance system wherein 
the cameras are capable of automatically adjusting certain operational parameters, such as the 
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focus setting, of each of the cameras. Additionally, it would be desirable to provide a 
multiple-camera surveillance system wherein manual adjustments of several camera 
parameters can be made remotely, stored, and then retrieved in association with the 
reorientation of the camera. 
5 In particular, it would be desirable to provide such storage and retrieval of 

parameters in connection with the focus and/or white balance of a colour video camera. In 
order to obtain accurate colour images from a video camera, it is necessary to equalize, or 
balance, the levels of red, green, and blue in a video signal relative to the detected levels of 
such colours, in accordance with the spectrum of the light in the observed area. Sunlight, 
10 for example, has a spectrum that approximates a 5,500 K blackbody, hence the spectrum of 
reflected light from a white object will exhibit a peak in the green region. The spectra of 
incandescent lamps can be approximated by blackbody spectra at various temperatures 
depending upon the type of lamp. Still other sources of light, such as metal vapour lamps or 
fluorescent tubes, exhibit prominent peaks at various locations within the spectrum. A figure 
15 of merit that is used to characterize the spectrum of a light source is referred to as the colour 
temperature of the light source. In the field of photography, coloured filters are used to 
compensate, or equalize, the response of colour film to account for various colour 
temperatures. In electronic imaging systems, it is common to provide a variable gain device, 
known as a white balance system, to allow the user to equalize the response of an electronic 
imaging device in accordance with the prevailing colour temperature. 

In some applications the remote surveillance unit is employed in a location 
having different areas each of which can be characterized by a colour temperature. For 
example, one area within a room may have incandescent lighting, while another area of the 
room may have fluorescent lighting, or a window providing sunlight. Automatic white 
balance systems are known wherein the levels of compensation applied to the colour 
components of a video image are continuously adjusted in response to the measured coloured 
content of the image. Such white balance systems each rely upon a statistical assumption in 
regard to what the colour content of a balanced image should be. If, for example, there is 
relatively little white actually in an area being viewed, an automatic white balance system 
may incorrectly distort the colours in the video image. Additionally, the continuous colour 
adjustment provided by automatic white balance often causes colour instability within the 
video image, particularly if there are fluctuations in the colour content of the area as may be 
caused by movement of the camera or objects in the image. 
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White balance settings can be adjusted manually by the camera operator, but 
manual adjustment would be a distraction to the user of a video surveillance system, 
especially a system wherein a plurality of remotely positionable cameras are employed. It 
would be desirable to provide a video surveillance system in which predefined white balance 
5 settings can be stored in association with corresponding areas that may be viewed by each 
camera. Then, these settings can be automatically retrieved, and applied to the detected 
video image whenever the camera is reoriented to view one of the selected areas, which may 
have a different colour temperature than a previously viewed area. 

In some applications, it is desirable to provide the surveillance system user with 

10 the ability to set or adjust the shutter speed of a remote surveillance camera. For example, 
in a dimly illuminated area, a low shutter speed is generally desirable. Whereas, in a 
brightly illuminated area, a relatively high shutter speed is generally desirable. Additionally, 
a surveillance camera may be employed in an area where it is desirable to obtain a clear 
image of moving objects, e.g., vehicles, that may pass through the area. It would be 

15 desirable to provide the user with the ability to set or adjust the shutter speed of a remote 
surveillance camera depending upon the circumstances presented in the monitored area. 

SUMMARY OF THE INVENTION 

A surveillance system is provided wherein a plurality of remote surveillance 

20 units are arranged to receive commands from a user-operated central processing unit. Each 
surveillance unit is provided with a video camera and a pan/tilt mechanism for positioning 
the camera in response to commands received from the user. Each surveillance unit is 
further provided with a microcontroller for controlling the camera and the pan/tilt mechanism 
in response to commands received from the user. The microcontroller is also operative to 

25 store and to retrieve certain camera settings, in association with selected orientations of the 
pan/tilt mechanism, in response to commands from the user to memorize or to recall such 
information. 

A remote surveillance unit is connected with a local source of AC power for 
providing electrical power to the surveillance unit. The remote surveillance unit includes 
30 circuitry for generating a line reference signal that is synchronized with the local AC source. 
An imaging controller receives the line reference signal and produces a synchronization 
reference signal which is used to generate synchronization signals within a composite video 
signal produced by the camera. The image controller produces a selected degree of phase 
lag in the synchronization reference signal relative to the line reference signal. The degree 
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of phase lag may be remotely selected by the user in order to effect synchronization of 
composite video signals received from a plurality of remote surveillance units. An automatic 
phase adjustment circuit is provided for detecting lack of synchronization among the 
displayed video images and for automatically adjusting the relative phase lag among the 
5 remote surveillance units. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The foregoing summary as well as the following detailed description of a 
preferred embodiment of the present invention will be better understood when read in 
10 conjunction with the appended drawings in which: 

FIG. 1 is a functional block diagram of a video surveillance system; 

FIG. 2 is a functional block diagram of a remote surveillance unit of the 
surveillance system of FIG. 1; 

FIG. 3 is a partial schematic diagram showing certain communication and 
15 control connections among elements of the remote surveillance unit of FIG. 2; and 

FIG 4 is a partial schematic diagram of an arrangement for automatically 
detecting and adjusting the relative phase of the vertical synchronization component of 
composite video signals received from a plurality of remote surveillance units. 

20 DETAILED DESCRIPTION 

Referring now to FIG. 1, there is shown a video surveillance system 10. The 
surveillance system 10 includes a plurality of remote surveillance units 12a, 12b, and 12c. 
The surveillance units 12a, 12b, and 12c are of the type disclosed in commonly-owned, co- 
pending application entitled "Surveillance Camera System", filed concurrently herewith and 

25 incorporated by reference herein. Each of the surveillance units 12a, 12b, and 12c is 

configured as shown in reference to surveillance unit 12a, and includes a video camera 14. 
The camera 14 is mounted within a translucent hemispherical dome that is preferably 
attached to, or suspended from, a ceiling at the location in which the surveillance unit 12a is 
deployed. The camera 14 is attached to a universal mounting (not shown) which is capable 

30 of orienting the camera at a selected horizontal angle (or pan coordinate) by a pan motor 16, 
and at a selected vertical declination (or tilt coordinate) by a tilt motor 18. Additionally, the 
pan motor 16 and the tilt motor 18 are operative to vary the pan and tilt coordinates of the 
camera at one or more selected angular velocities. 



BNSDOCID: <WO. 



9535627A1_I_> 



WO 95/35627 PCT/1B 95/00501 

5 

Each of the cameras within the respective surveillance units 12a, 12b, and 12c 
produces a video signal that is transmitted, by respective video cables 22a, 22b, and 22c, to 
a video switcher/multiplexer 24. The video switcher/multiplexer 24 is preferably of the type 
which is operative to receive a plurality of video signals, such as from video cables 22a, 
5 22b, and 22c, and to cause the corresponding images to be displayed upon one or more video 
monitors 26a, 26b, and 26c. For example, the respective images may each be shown on an 
assigned monitor. Alternatively, two or more of the respective images may be displayed on 
a single monitor by using a selected-image, switching sequence. 

The manner in which the switcher/multiplexer 24 displays incoming video 

10 signals upon the monitors 26a, 26b, and 26c is determined by control signals that are 
provided to the switcher/multiplexer 24 by a central processing unit, CPU 28, which is 
connected to the switcher/multiplexer 24. The CPU 28 is also connected with a user 
interface 30 that preferably includes a keypad 32 for providing alpha-numeric information to 
the CPU 28, a plurality of function keys 34 for causing the CPU 28 to execute pre- 

15 determined procedures, an alpha-numeric display 36 for displaying information pertaining to 
the status of the surveillance system 10, and an analog input device such as a joystick 38 for 
providing analog positioning information to the CPU 28. 

By pressing an appropriate sequence of function keys 34 and/or alpha-numeric 
keys 32 during operation of the surveillance system, a user can inform the CPU 28, of the 

20 desired image, or sequences of images, to be displayed by the switcher/multiplexer 24 upon 
the monitors 26a, 26b, and 26c. The CPU 28 is programmed to cause an appropriate prompt 
to appear in the display 36 to assist the user in entering such information. Then, the CPU 28 
downloads the entered information to a sequencer within the switcher/ multiplexer 24 so that 
the video images will be displayed in the desired manner. Alternatively, the CPU 28 is 

25 programmed to control the switching and display functions of the switcher/multiplexer in 

"real-time" as the operation of the surveillance system proceeds. The user interface 30, CPU 
28 and switcher/multiplexer 24 are preferably of the type manufactured and sold by Burle 
Industries of Lancaster, Pennsylvania as the ALLEGIANT video switching system. 

In addition to controlling the switcher/multiplexer 24, the CPU 28 also provides 

30 the user with the ability to issue a variety of commands from the keyboard to the remote 
surveillance units 12a, 12b, and 12c. Particular ones of such commands and the manner in 
which such commands are effected within the surveillance system shall be described 
hereinbelow in greater detail. In general, the user specifies a command by operating the user 
interface 30 according to a pre-determined sequence, such as pressing a function key 34 to 
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select a command, and then specifying a camera, via the keypad 32, to which the command 
pertains. Such commands are accompanied by additional user inputs, such as numeric 
parameters or joystick positions, that are pertinent to execution of the commands. Upon 
detecting that a command is to be sent to a specified surveillance unit, the CPU 28 provides 
5 a corresponding command signal to a signal distribution unit, SDU 40. 

The SDU 40 receives command signals from the CPU 28 and, in turn, 
broadcasts a serial-encoded, digital command word to the surveillance units 12a, 12b, and 
12c via respective twisted-pair cables 42a, 42b, and 42c. Each remote surveillance unit 12a, 
12b, and 12c includes a dome control system 20 (i) for receiving command words over the 
10 associated twisted-pair cable, (ii) for decoding the command words received from the 
associated twisted-pair cable, and (iii) for effecting an appropriate response within the 
surveillance unit. 

Each dome control system 20 is provided with a unique address code by which 
the SDU 40 identifies the surveillance unit for which a particular command is intended. For 

15 example, each command word issued by the SDU to the surveillance units includes a 

command prefix, comprising an attention signal and the address code of the surveillance unit 
for which the command word is intended. Then, each dome control system 20 responds only 
to command words which include the correct command prefix. The remaining portion of the 
command word includes a command code identifying the command, and one or more 

20 parameters associated with the command. 

In an alternative embodiment, the address code of each dome control system is 
used to identify a frame interval during which the dome control system will respond to 
commands issued by the SDU 40. In such an embodiment, the SDU 40 issues command 
words during distinct frame intervals between periodic synchronization signals. Each dome 

25 control system then responds only to command words within its assigned interval. Various 
other arrangements for delivery of commands from the CPU 28 to the remote surveillance 
units are possible within the scope of the present invention. 

Representative command words that are issued to a remote surveillance unit are 
the commands listed in Table 1. Each of the commands in Table 1 is represented by a 

30 unique digital signal. Certain ones of the commands in Table 1 are broadcast by the SDU 40 
in conjunction with one or more numeric or logical arguments as indicated in Table 1. 
Alternatively, any of the commands which take a logical argument, such as AUTO-FOCUS 
< ON/OFF >, may be implemented as two separate commands which each indicate one of 
the conditions of the argument, for example AUTO-FOCUS-ON and AUTO-FOCUS-OFF. 
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FUNCTION 
MEMORY: 

MOTION: 



LENS CONTROL: 



< OPEN/CLOSE > 



IMAGING: 



30 < INCREASE/DECREASE > 
< INCREASE/DECREASE > 



TIMING: 



Table 1 
COMMAND 



PRESET 
RECALL 



PAN 
TILT 
ZERO 



AUTO-FOCUS 
FOCUS 
AUTO-IRIS 
IRIS 



ZOOM 



SHUTTER 

AUTO-WHITE 

RED 

BLUE 

WHITE-SET 



ARGUMENT 



<#> 
<#> 



< VELOCITY > 

< VELOCITY > 



< ON/OFF > 

< IN/OUT > 

< ON/OFF > 



< IN/OUT > 



< SPEED > 
< ON/OFF > 



LINE-PHASE 

< INCREASE/DECREASE > 

The actions that are effected within a remote surveillance unit in response to 
receiving the commands listed in Table 1, pertain to the general categories of motion control, 
40 lens control, and image control. The PRESET command provides the user with the ability to 
define and to store a combination of two or more camera control parameters. The RECALL 
command allows the user to instruct each of the surveillance units to operate according to the 
stored parameters. 

Each of the commands listed in Table 1 shall now be described with reference 
45 to FIG. 2 wherein there is shown a more detailed diagram of a surveillance unit 12a. 
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Command words issued from the SDU 40 are received via twisted-pair cable 
42a at a receiver/decoder 44. The command words received over twisted-pair cable 42a are 
preferably encoded in a bi-phase, or pseudoternary serial code. The receiver/decoder 44 
converts the received command words into a format, such as 5-volt TTL, which is then 
5 provided to a dome microcontroller 46. The dome microcontroller 46 preferably includes an 
MC68HC16 microcontroller manufactured by Motorola, Incorporated. 

Upon receipt of a command word, the microcontroller 46 first compares the 
address prefix of the command word to an address code assigned to the remote surveillance 
unit 12a. The address code is assigned to the surveillance unit 12a according to the position 

10 of a thumbwheel switch (not shown) or by a similar device for providing a code to the dome 
microcontroller 46. If the received command prefix does not match the pre-determined 
address of the surveillance unit 12a, then the command is ignored and no further action is 
taken by the dome microcontroller 46 in response to the received command word. If the 
received command prefix matches the pre-determined address of the surveillance unit 12a, 

15 then the dome microcontroller 46 responds to the received command word, and any 
parameters associated therewith, in accordance with instructions stored in EPROM 48. 

In alternative embodiments, the predetermined addresses may be assigned to the 
remote surveillance units in various ways, such as DIP switch settings or numerical values 
stored in predefined non-volatile memory locations. 

20 The specific actions taken by microcontroller 46 in response to a received 

command word intended for the surveillance unit 12a shall now be described in greater 
detail. 

Camera Motion 

25 The dome microcontroller 46 is programmed to control the pan and tilt motors 

in response to the PAN and TILT commands and to maintain a record of the instantaneous 
angular position of the camera. Referring back to FIG. I, the user may press an appropriate 
function key 34 and a numeric key 32 to inform the CPU 38 that the user desires to adjust 
the angular coordinates of the camera 14, within surveillance unit 12a. The CPU 28 then 

30 displays an appropriate message on display 36 informing the user that the specified camera 
has been selected for such control. The CPU 28 periodically samples the X and Y positions 
of joystick 38. The user may then deflect the joystick 38 in the X and/or Y directions in 
order to indicate the desired direction of angular motion of the camera in the respective pan 
and tilt directions. Additionally, the desired speed of such angular motion can be specified 



BNSDOCtD: <WO 9535627A1_I_> 



WO 95/35627 PCT/IB95/00501 

9 

by the user according to the extent to which the joystick is deflected in the desired direction. 
The CPU 28 also selects, for each axis of movement, a digital value or values representing 
the desired angular velocity (i.e., the speed and direction) in each of the pan and tilt axes. 

For example, if the joystick is deflected only in the X direction, the CPU 28 
5 selects a digital code representing the desired pan velocity. Then the CPU 28 issues a PAN 
command to the SDU, along with the velocity code. 

If the joystick is moved in the X and Y directions simultaneously, then the CPU 
28 selects a first digital code representing the desired pan velocity, and a second digital code 
representing the desired tilt velocity. Separate PAN and TILT commands are then provided 
10 to the SDU 40 for transmission to the surveillance units. The SDU 40 then broadcasts the 
corresponding command words successively. 

Referring again to FIG. 2, the dome microcontroller 46 responds to receipt of 
the PAN or TILT commands by issuing a corresponding control signal to pan motor driver 
50 or to tilt motor driver 52. Pan motor driver 50 is a motor control circuit that receives a 
15 control signal from the dome microcontroller 46 and in response thereto causes pan motor 16 
to operate so as to pan the camera at the desired pan velocity. Similarly, tilt motor driver 52 
receives a control signal from dome microcontroller 46 and in response thereto causes tilt 
motor 18 to operate so as to tilt the camera at the desired tilt velocity. 

The selected pan and tilt velocities are maintained by the respective drive 
20 circuits 50 and 52 until new values are received from the microcontroller 46. For example, 
when the user releases the joystick 38, the dome microcontroller 46 will in due course 
receive appropriate PAN and TILT commands from the SDU 40 indicating that the pan and 
tilt velocities are to be set to zero. At such time, the dome microcontroller 46 updates the 
respective pan and tilt control signals provided to motor drivers 50 and 52 in order to stop 
25 motors 16 and 18. 

Pan motor 16 and tilt motor 18 are preferably stepper motors of the well-known 
type which provide accurate open-loop positioning and are controllable by well-known digital 
circuitry. In an alternative embodiment, another type of motor, such as a servomotor, is 
employed. 

30 In addition to providing control signals to the pan and tilt drivers 50 and 52, the 

dome microcontroller 46 continuously computes and maintains a record of the instantaneous 
angular coordinates of the camera. Since the pan and tilt motors are preferably driven in an 
open-loop manner, the dome microcontroller 46 is programmed to execute an initial 
"homing" procedure when the surveillance unit is initially powered. During the homing 
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procedure, the pan and tilt motors are positioned to a pre-determined pair of angular 
coordinates so that the microcontroller 46 can than properly compute the instantaneous 
position of the pan and tilt motors by "dead reckoning". For example, the dome 
microcontroller 46 may maintain a count of the impulses applied to the stepper motors during 
5 movement of the camera. Alternatively, the dome microcontroller may integrate the velocity 
values obtained during operation of the surveillance system. 

Because the pan and tilt motors are preferably controlled in an open-loop 
manner, positional errors may accumulate in the computed angular coordinates relative to the 
actual angular coordinates of the camera. In order to eliminate such errors, the user may 

10 initiate execution of the homing procedure using the ZERO command. Alternatively, or in 
addition thereto, the pan and tilt mechanism may be provided with appropriate sensors 
connected with the dome microcontroller 46, for detecting when the camera is in a 
predetermined orientation. Then, the computed angular coordinates may be automatically re- 
ZEROED whenever the positional sensors are actuated. 

15 The instantaneous angular coordinates of the camera are stored within internal 

registers of the dome microcontroller 46. Whenever a PRESET command is received by 
dome microcontroller 46, the instantaneous angular coordinates are stored in the EEPROM 
56 at a location determined by the numeric argument of the PRESET command. Whenever a 
RECALL command is received by the dome microcontroller 46, the stored coordinates are 

20 retrieved from the EEPROM 56 and compared with the instantaneous coordinates of the 
camera. Then, the dome microcontroller 46 computes the required angular velocities in the 
pan and tilt directions to effect positioning of the camera according to the retrieved 
coordinates. After the required velocities have been computed, the dome microcontroller 46 
actuates the pan and tilt drives 50 and 52 in order to position the camera in accordance with 

25 the retrieved coordinates. 

The numeric argument associated with the PRESET and RECALL commands 
may be used to specify which of a plurality of stored coordinates are to be stored and 
retrieved. The user may specify the argument of the PRESET and RECALL commands as a 
decimal integer which is converted by the dome microcontroller 46 into a base EEPROM 

30 address for retrieving the stored coordinates and any other parameters that may be stored in 
response to a PRESET command. The coordinates that are retrieved by a RECALL 
command are the same coordinates which were stored previously in response to a previous 
PRESET command having the same numeric argument. The CPU 28 keeps a record for 
each surveillance unit of how many sets of coordinates have been stored. The CPU 28 is 
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programmed to ignore the command, if the user specifies an undefined RECALL argument. 
Alternatively, the dome microcontroller 46 is programmed to ignore RECALL commands 
having an undefined argument. 



5 Lens Control 

The camera 14 includes a lens system 58 and an imaging system 60. The lens 
system 58 includes a zoom lens 62 having an adjustable iris. The lens system 58 also 
includes lens controller 64 which provides electro-mechanical control of the iris opening, 
focus, and zoom factor of the lens. A preferred lens system 58 is the model 55YD lens 

10 system manufactured by Tamron Corporation of Japan. 

Zoom lenses are known which include a complex mechanical arrangement 
whereby the effective iris opening is automatically increased when the zoom factor of the 
lens is increased. Because such lenses tend to be relatively bulky, their use in remote 
surveillance units can limit the speed at which a camera can be positioned in a desired 

15 orientation. Additionally, such lenses require a larger dome housing than would be desirable 
in an unobtrusive surveillance unit. We have found that the use of a lens which does not 
provide such mechanical aperture compensation, such as the preferred lens mentioned above, 
allows a significant size reduction to be obtained in the construction of a remote surveillance 
unit and provides rapid positioning of the camera. 

20 The lens controller 64 is connected with the dome microcontroller 46 via a 

four-wire synchronous serial connection 63. Communication from the dome microcontroller 
46 to the lens controller 64 is conducted using a four-byte control token that includes a 
control byte, a zoom byte, a focus byte, and an iris byte. The control byte contains a code 
by which the lens controller 64 is instructed to operate according to one or more modes, 

25 including an auto-focus mode, and auto-iris mode, or an external control mode. 

In the external control mode, the lens controller 64 responds to the control 
token by setting the iris opening, the focal distance, and/or the zoom factor of the lens in 
accordance with values contained in the iris byte, the focus byte, and the zoom byte, 
respectively, of the control token. 

30 In the auto-focus mode, the lens controller 64 utilizes an internal feedback 

mechanism to adjust the focus of the lens. The video signal produced by the imaging system 
60 is supplied as an input to the lens controller 64 to allow the auto-focus mechanism to 
adjust the focus of the lens to maximize the degree of contrast in the video signal. 
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In the auto-iris mode, an internal control mechanism of the lens controller is 
employed to adjust the iris opening of the lens in order to maintain the amplitude of the 
video signal within a pre-determined range. The auto-iris mode of operation can be 
employed in combination with, or independently of, the auto-focus mode of operation. 
5 In addition to receiving and responding to a control token from the dome 

microcontroller 46, the lens controller 64 is operative to issue a reply token to the dome 
microcontroller 46 when the lens controller receives a control token. The reply token is 
similar to the control token, and contains data fields into which the then-current status of the 
lens 62 is encoded. In order to obtain the current lens settings, the dome microcontroller 46 
10 transmits to the lens controller a control token including a control byte that causes the lens 
controller 64 to issue a reply token without changing any of the current lens settings. 

It will be readily appreciated that the incorporation of an automatic lens 
focusing mechanism into a video surveillance system greatly relieves the burden upon the 
user of the surveillance system who would otherwise expend considerable attention upon the 
15 task of adjusting the focus for each of multiple cameras. 

The keyboard is equipped with an appropriate function key by which the 
operator signals to the CPU 28 to select manual focusing in connection with a particular 
camera. The CPU 28 is programmed to cause the SDU 40 to issue the AUTO-FOCUS 
< ON/OFF > command word to the indicated surveillance unit, e.g., surveillance unit 12a. 
20 The dome microcontroller 46 then receives the AUTO-FOCUS command and issues a control 
token to the lens controller in order to activate or to deactivate the auto-focus function as 
selected by the user. The keyboard is also equipped with an appropriate mechanism, such as 
a knob (not shown), by which the user indicates, in manual focus mode, whether the focal 
distance is to be increased or decreased. During manual focus operation, the CPU causes the 
25 SDU 40 to periodically issue FOCUS < IN/ OUT > command words to the selected camera 
in response to the user's manipulation of the focusing knob. As each FOCUS command is 
received by the dome microcontroller 46, the dome microcontroller 46 obtains the current 
focus value from the lens controller 58 and then issues a control token to the lens controller 
wherein the value of the focus byte is increased or decreased in accordance with the user's 
30 action. 

As has been discussed previously, the dome microcontroller 46 operates to 
record selected angular coordinates and to rapidly position the camera in response to the 
RECALL command. It may occur in a given application that one such recorded orientation 
has been selected by the user to provide a view of an area that is close to the camera, and 
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another recorded orientation has been selected to provide a view of a second area that is 
relatively more distant from the camera. Contemporaneously, it may happen that the relative 
distances from the camera are such that different focal positions of the lens would be 
required to provide a sharp image in the respective recorded orientations. If the camera is 
5 being operated in the manual focus mode, the operator would be required to adjust the focus 
of the lens each time that the RECALL command is issued in order to alternately view the 
two selected areas. If the camera is being operated in the auto-focus mode, then a time delay 
occurs while the auto-focus mechanism adapts to each view whenever the RECALL 
command is issued. Depending upon the relative distances to the objects of interest and the 

10 nature of the image content of the areas of interest, the adaptation time of the auto-focus 
mechanism could be sufficiently long to annoy the user or to cause the user to miss an 
important event in one of the selected areas. 

In order to provide more rapid focusing of the lens, the dome microcontroller 
46 is programmed to store the then-current focus byte value into the EEPROM 56 whenever 

15 the PRESET command is received. The focus byte is stored in association with the angular 
coordinates that are also stored in response to the PRESET command as described previously 
herein. Subsequently, whenever a RECALL command is received by the dome 
microcontroller 46, the focus value pertaining to the specified preset can then be retrieved 
and issued to the lens controller by the dome microcontroller in a control token. In this 

20 manner the camera is positioned in the desired orientation and properly focused in a more 
rapid manner than previously known because the need to re-focus the camera or to wait for 
adaptation of the auto-focus mechanism is eliminated. 

The AUTO-IRIS, IRIS, and ZOOM commands operate upon the function of the 
lens system in order to set or adjust the iris and zoom parameters of the lens system 58 in 

25 the same manner as has been described in connection with the focus parameter. 

According to the procedures discussed in connection with storing and recalling 
selected focal settings in response to the PRESET and RECALL commands, the dome 
microcontroller is further programmed to store and recall the current values of the iris byte 
and/or the zoom byte of the lens reply token in response to the respective PRESET and 

30 RECALL commands. Hence, the operational parameters pertaining to the lens system can be 
rapidly retrieved and effected for each of a plurality of preselected angular coordinates. 

In an alternative embodiment, the user specifies which of the lens parameters 
are to be stored and recalled in response to the PRESET and RECALL commands. For 
example, the desired combination of lens parameters is selected via the keyboard and then 
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included as an additional argument of the PRESET command. If a subset of the possible 
lens parameters is stored as a preset, then the EEPROM locations that would otherwise be 
occupied with the non-selected parameters are provided with a null, or "don't care", code. 
In such an embodiment, the dome microcontroller 46 responds to a RECALL command by 
5 providing a control token to the lens controller containing the retrieved parameters in the 
selected fields, and the then-current parameters in the non-selected fields of the control 
token. 

Imagine 

10 The imaging system 60 of the camera 14 includes an imaging device, such as a 

charge-coupled device (CCD) 66, positioned behind the lens 62. The CCD 66 is preferably 
a Sony ICX054AK 1/3-inch colour CCD array. The CCD 66 is electrically driven by a 
driver circuit 68 which provides the necessary timing signals to operate the CCD 66. The 
driver circuit preferably includes a Sony CXD1257AR CCD camera timing generator in 

15 addition to other circuitry for generating horizontal and vertical synchronization signals used 
in producing a video signal. The synchronization signals produced by the CCD driver 68 are 
provided to a video processing subsystem 70. The video processing subsystem 70 includes 
sampling circuitry for extracting signals from the CCD 66, processing circuitry for adjusting 
image features such as white balance, and an encoder for converting the image into a 

20 standard composite video format, such as the NTSC format or the PAL format. The imaging 
system 60 further includes an imaging controller 72. The imaging controller 72 
communicates with the dome microcontroller 46 via a two-wire asynchronous serial TTL 
connection 71. The imaging controller 72 provides control signals to the CCD driver 68 for 
establishing a time interval used by the CCD driver 68 to control the integration time, or 

25 shutter speed, of the CCD 66. 

In order to provide the user with the ability to remotely set or adjust the shutter 
speeds of the various cameras in the surveillance system 10, the CPU 28 is programmed to 
recognize shutter speed commands, entered by the user upon the user interface 30, and then 
to relay those commands to the SDU 40. For example, the user specifies a particular camera 

30 and the shutter speed of the specified camera by pressing appropriate keys on the keyboard. 
Upon receiving an appropriate signal from the CPU 28, the SDU 40 broadcasts the 
SHUTTER < SPEED > command word to the surveillance units along with the proper 
command prefix. 
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Referring back to FIG. 2, the SHUTTER command is received, via twisted- 
pair 42a, by the receiver 44 and then provided to the dome microcontroller 46. The dome 
microcontroller 46 is programmed to respond to the SHUTTER command by requesting the 
current shutter speed setting from the imaging controller 72, which maintains a record of the 
5 current shutter speed. The dome microcontroller 46 then issues a control signal on the serial 
connection 71 indicating that the shutter speed is to be set to the newly-received value. 

Referring now to FIG. 3, the serial connection 71 includes a transmit line, TX, 
and a receive line, RX. The transmit line TX provides transmission of serial data from the 
dome microcontroller 46 to the imaging controller 72. The receive line RX provides 

10 transmission of serial data from the imaging controller 72 to the dome microcontroller 46. 
The receive line RX is also connected with an internal shift register 74 of a CCD camera 
timing generator 68a, which is part of the CCD driver circuitry 68. One such timing 
generator is the Sony CXD1257AR. The shift register 74 of the timing generator 68a is 
connected to an internal gating mechanism of the timing generator 68a so that the data held 

15 in the shift register 74 is gated into a shutter speed register of the timing generator 68a 

whenever a strobe pulse is applied to the STB input of the timing generator 68a. When the 
imaging controller 72 detects that a shutter speed setting has been transmitted on signal line 
TX by the dome microcontroller 46, the shutter speed is then transmitted by the imaging 
controller 72 upon the RX line, and followed by a pulse applied to the STB line by the 

20 imaging controller 72. 

The imaging controller 72 maintains a record of the current shutter speed, and 
updates the current shutter speed each time that the shutter speed is adjusted. The imaging 
controller is further programmed to send the current shutter speed setting to the dome 
microcontroller 46, via signal line RX, whenever such information is requested by the dome 

25 microcontroller 46. 

Since the dome microcontroller 46 obtains and sets the current shutter speed of 
the camera, the shutter speed can be included among the parameters that are stored when a 
PRESET command is issued. The stored shutter speed values can be retrieved by the dome 
microcontroller 46 and then issued to the imaging system when a RECALL command is 

30 received by the dome microcontroller 46. Thus, the user is provided with the ability to 

cause the dome microcontroller to record a plurality of camera configurations each having an 
associated shutter speed so that such configurations may be recalled in swift response to 
changing lighting conditions or in response to movement within the monitored area. 
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The white balance of the video image is determined by control inputs RCNT 
and BCNT of a colour processor 70a, which is part of the video processing subsystem 70. 
In the preferred embodiment, the colour processor 70a is a Sony CXA1391Q/R processor 
that requires a 0-5 volt analog voltage on the RCNT and/or BCNT inputs in order to specify 
5 the levels of red and/or blue gain to be applied to the signal from the CCD relative to the 
green level. In order to provide such analog control voltages to the colour processor 70a 
from the imaging controller 72, a pair of digitally-controlled pulse-width modulators 72a and 
72b are employed within the imaging controller 72 to produce square waves having duty 
cycles that are proportional to the respective desired analog control voltages (as a proportion 
10 of the supply voltage). The duty cycles of the pulse width modulators 72a and 72b are 
controlled by internal data registers of the imaging controller 72. 

The square wave from pulse-width modulator 72a is connected with an RC 
network 76a. The RC network 76a averages the square wave from the pulse-width 
modulator 72a, and the resulting average voltage is applied to the BCNT input of the colour 
15 processor 70a. Similarly, the square wave from pulse-width modulator 72b is averaged by 
RC network 76b and then provided to the RCNT input of colour processor 70a. 

The numerical values which are used to regulate the duty cycle of pulse-width 
modulators 72a and 72b are determined by imaging controller 72 as follows. If the user 
desires to have the white balance of the camera continuously and automatically adjusted, the 
20 user specifies such operation by pressing an appropriate function key on the user interface. 
Such a request causes the AUTO- WHITE command word to be received by the dome 
microcontroller 46. The dome microcontroller 46 informs the imaging controller 72, via 
signal line TX, that such a request for automatic white balancing has been received. 

As can be seen in FIG. 3, the colour processor 70a provides several analog 
25 output signals, collectively designated RGB. The RGB outputs of the colour processor 70a 
are connected with an analog-to-digital converter 72c of the imaging controller 72. The RGB 
signals are representative of the spatial average of the respective red, green, and blue 
amplitude levels in the image detected by the CCD 66. The imaging controller 72 is 
programmed to execute a feedback control procedure whereby the duty cycles of the pulse- 
30 width modulator 72a and 72b are adjusted to obtain a pre-determined relationship among the 
average levels of red, green, and blue in the detected image and as provided to the imaging 
controller 72 by the A/D converter 72c. Such a feedback control mechanism requires a 
certain amount of adaptation time in order to properly adjust the white balance when the 
camera is rapidly oriented toward areas having different colour temperatures. Additionally, 
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false adaptation of the white balance control mechanism can occur if the colour content of the 
monitored area does not exhibit the assumed statistical relationship among the red, green, 
and blue levels which the feedback mechanism is programmed to attain. 

In order to avoid the last-mentioned potential difficulties in obtaining and 
5 maintaining an accurate white balance, the surveillance system is preferably arranged to 

allow the user to manually specify the desired levels of red and blue compensation using the 
respective increase RED and BLUE commands. When these commands are received by the 
dome microcontroller 46, they are then relayed to the imaging controller 72 via the TX line. 
The imaging controller 72 then increments or decrements the appropriate control register 

10 pertaining to the pulse-width modulator 72a or 72b responsible for the specified red or blue 
gain. In order to substantially eliminate adaptation time or false adaptation from the white 
balance control mechanism, the control parameters used to drive the pulse width modulators 
72a and 72b, whether set automatically or manually, are preferably included among the other 
parameters that are stored by the dome microcontroller 46 in response to a PRESET 

15 command. For example, when a PRESET command is received by the dome microcontroller 
46, the dome microcontroller 46 subsequently issues a request via signal TX, for the imaging 
controller 72 to obtain the current values of the control parameters governing each of the 
pulse-width modulators 72a and 72b. These parameters are then sent to the dome 
microcontroller 46 via the RX connection and subsequently stored in the EEPROM in 

20 association with the other parameters stored in response to a PRESET command. 

Subsequently, when a corresponding RECALL command is received, the white balance 
setting is retrieved by the dome microcontroller 46 and transmitted to the imaging controller 
72 via the TX line. The imaging controller then adjusts the pulse- width modulators 72a and 
72b in order to restore the memorized white balance levels. 

25 In addition to the RED and BLUE commands, the WHITE-SET command may 

be specified by the user in situations wherein it is desirable to momentarily actuate white 
balancing and subsequently to retain the RED and BLUE gain settings. For example, rather 
than to manually adjust each of the RED and BLUE gain settings, the user may position the 
camera to view a white object or area within the monitored location. Then, the user may 

30 specify the WHITE-SET command. This command is then relayed from the CPU 28, and 
the SDU 40, to the remote surveillance units 12a-c. The dome microcontroller within the 
specified remote surveillance unit will then issue a command to the associated imaging 
controller to perform momentary automatic white balancing. 
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Momentary automatic white balancing is accomplished by the imaging controller 
by executing the automatic white balancing procedure for a predetermined period of time, 
and then maintaining the resulting control values within the control registers associated with 
the pulse-with modulators. 

5 As can be appreciated, the surveillance system of the present invention provides 

the ability to store and to recall a variety of imaging parameters in association with selected 
angular coordinates of the camera. Other parameters in addition to the ones discussed 
hereinabove may also be stored and recalled in a similar manner. For example, the 
preferred processing circuitry can be set to provide exposure compensation for scenes in 

10 which the subject of interest is illuminated from behind the subject relative to the camera. 
Such "backlight compensation" can be requested by the user in a manner similar to that 
described in connection with automatic white balance. Additionally, the presence or absence 
of backlight compensation may be among the parameters stored and/or recalled in connection 
with the PRESET or RECALL commands. 

15 

Timing 

As shown in FIG. 2, the dome control system 20 includes a power supply 
circuit 80, comprising several voltage regulators, for supplying electrical power to the other 
components of the dome control system 20 and to the pan motor 16, the tilt motor 18, the 

20 lens system 58, and the imaging system 60. The power supply circuitry 80 is connected, via 
transformer 86, to a source of AC power that is local to the remote surveillance unit 12a. 

Associated with the power supply circuitry is a linelock reference generator 82. 
The linelock reference generator 82 receives an AC signal from the transformer 86 and 
preferably includes a zero-crossing detector for generating a ll incompatible linelock 

25 reference signal that is synchronized to the AC signal from the transformer 86. The linelock 
reference signal from the linelock reference generator 82 is provided, via signal line 84, to 
the imaging controller 72. The imaging controller 72 is programmed to detect signal 
transitions on line 84 and, in response thereto, provide a synchronization reference signal to 
a phase-locked loop in the timing circuitry 68. Such a phase-locked loop is provided within 

30 the timing circuitry 68 in order to drive a vertical synch generator which produces the 

vertical synch signals for the CCD driver of the timing circuitry 68, and the composite video 
encoder of the video processor 70. 

The imaging controller 72 utilizes an internal delay mechanism to provide the 
synchronization reference signal to the timing circuitry 68 in accordance with a selected 
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degree of phase lag relative to the input signal on signal line 84 from the linelock reference 
generator 82. The degree of phase lag applied to the synchronization reference signal is 
determined by a delay value contained in an internal register of the imaging controller. 
Thus, the relative phase lag between the local AC power supply 88 and the vertical synch 
5 component of the composite video signal produced by the video processor 70, is determined 
by the delay value. 

In the surveillance system 10 of FIG. 1, wherein video cameras are employed at 
a plurality of remote locations, it may happen that the vertical synch component of the 
composite video signals arriving at the video switcher/ multiplexer 24 are relatively out of 

10 phase. For example, the local AC power circuits serving the remote surveillance units may 
be provided by different branches of a three-phase AC power system in the monitored 
facility. An undesirable effect of such an out-of-phase condition is a momentary loss of 
vertical hold in a video monitor that is provided with a switched sequence of signals from the 
remote cameras by the video switcher/multiplexer. The resulting "roll" in the image on the 

15 video monitor can be distracting to the user of the surveillance system. In order to 

synchronize the remote surveillance units with respect to one another, the user presses an 

appropriate key sequence to cause the CPU 28 to issue the LINE-REF 

< INCREASE/DECREASE > command word, via the SDU 40, to a specified camera. 

Upon receiving the LINE-REF command word, the dome microcontroller 46 

20 instructs the imaging controller 72, via serial connection 71, to increase or decrease the value 
contained within the register responsible for determining the phase lag between the linelock 
reference signal produced by the linelock reference generator 82, and the synchronization 
reference signal provided to the timing circuitry 68. 

When the surveillance system is initially installed, the user can program the 

25 video switcher/multiplexer 24 to switch between two or more remote surveillance units and 
to provide the corresponding images alternately upon a single video monitor. Then, by 
watching the monitor, the user can adjust the relative phase of the received composite video 
signals until no image roll is observed when switching occurs. After adjusting the relative 
phase between one pair of composite video signals, the user adjusts the phase of the 

30 composite video signals from the remaining remote surveillance units in a similar manner. 
In order to provide greater phase adjustment accuracy and to relieve the operator from 
manually adjusting the relative phase of each of several remote surveillance units, an 
automatic remote phase adjustment system is implemented as shown in FIG. 4 and described 
hereinbelow. 
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As has been described, the video switcher/ multiplexer 24 receives video signals 
from the remote surveillance units via signal lines 22a-c. The video switcher/ multiplexer 24 
includes switching circuitry 24a responsive to control signals from the CPU 28, for causing 
selected input video signals to be displayed upon monitors 26a-c in accordance with the pre- 
5 determined switching sequence. Video signals are provided from the switching circuitry 24a 
to the monitors 26a-c via respective signal lines 25a-c. 

Vertical synchronization stripper circuits 90a-c are connected to each of the 
corresponding video signal lines 25a-c. Each of the stripper circuits 90a-c is of the type 
which isolates the vertical synchronization component of a composite video signal. Hence, 

10 the stripper circuits 90a-c provide, on signal lines 91a-c, the vertical synchronization 
components of the video signals present on respective signal lines 25a-c. 

The synchronization signals on signal lines 91a-c are each connected to one 
input of respective phase comparators 94a-c. The other input of each of the phase 
comparators 94a-c is connected, via signal line 93, to a vertical synchronization generator 

15 92. The vertical synchronization generator 92 provides a reference vertical synchronization 
signal with which each of the signals on lines 91a-c is compared by the respective phase 
comparators 94a-c. The phase comparators 94a-c are preferably digital phase comparators of 
the type which provide a pulsed logic-level output signal indicating the direction and degree 
of phase difference between two input signals. Each of the phase comparators 94a-c 

20 produces a pulsed, digital output signal indicative of the difference between the reference 
vertical synchronization signal on line 93 and the respective vertical synchronization signals 
from the stripper circuits 90a-c. 

The output of the phase comparators 94a-c are each provided to respective 
averaging filters 96a-c. Each of the averaging filters 96a-c is formed to convert the pulsed 

25 digital signal, from the respective associated phase comparator 94a-c, into an analog voltage 
signal. The phase comparators 94a-c and the filters 96a-c are arranged so that, for example, 
if the reference vertical synchronization signal upon line 91a leads the vertical 
synchronization signal on line 93, then the filter 96a produces an analog voltage that is 
greater than one-half of the circuit supply voltage. Conversely, if the video synchronization 

30 signal on line 91a lags the reference vertical synchronization signal on line 93, then the filter 
96a produces an analog voltage which is less than one-half of the supply voltage. 

The analog voltage produced by filters 96a-c are provided to respective window 
comparators 98a-c. Each of the window comparators 98a-c is arranged to compare the input 
voltage with a pre-determined upper reference level and to produce a high logic-level signal 
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upon respective signal lines 99a-c, in the event that the input voltage exceeds the pre- 
determined upper reference level. In this manner, if the phase delay associated with the 
remote surveillance unit responsible for producing the composite video signal on line 25a 
must be increased in order to synchronize with the reference vertical synchronization 
5 generator 92, then a high logic-level would be present on line 99a. 

Conversely, each of the window comparators 98a-c is arranged to produce, on 
respective signal lines lOla-c, a high logic-level signal whenever the input voltage is below a 
pre-determined lower reference voltage. In this manner, the logic signals upon any of signal 
lines lOla-c represent whether it is necessary to decrease the phase delay within any of the 
10 remote surveillance units associated with the video signals present upon signal lines 25a-c, 
respectively. 

In order to provide the phase adjustment information upon signal lines 99a-c 
and lOla-c, to the CPU 28 in an efficient manner, signal lines 99a-c are connected as inputs 
to a multiplexer 100a, and signal lines lOla-c are connected as inputs to a multiplexer 100b. 
15 The address lines of respective multiplexers 100a and 100b are connected in parallel with the 
CPU 28. The CPU 28 then provides an address, on address lines 103, to the multiplexers 
100a and 100b, in order to determine whether a particular remote surveillance unit requires 
phase adjustment. 

For example, during operation of the video switcher/ multiplexer 24, the CPU 
20 may place, on address lines 103, an address which causes the output 99a of window 

comparator 98a to be gated to the output of multiplexer 100a (INC). Simultaneously, the 
output 101a of the window comparator 98a will be gated to the output of multiplexer 100b 
(DEC). The outputs INC and DEC of respective multiplexers 100a and 100b are provided as 
inputs to the CPU 28, so that the CPU 28 can determine, for a given address on lines 103, 
25 whether phase adjustment is required for the remote surveillance unit which is then 

producing the composite video signal on line 25a. Since the CPU 28 is also controlling the 
video switcher circuitry 24a, the CPU 28 can then issue the appropriate command to increase 
or decrease the phase delay in the identified remote surveillance unit. The command to 
increase or decrease the phase delay in the identified remote surveillance unit is then 
30 provided to the remote surveillance unit as has been described hereinabove. 

The terms and expressions which have been employed are used as terms of 
description and not of limitation. There is no intention in the use of such terms and 
expressions of excluding any equivalents of the features shown and described or portions 
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thereof. It is recognized, however, that various modifications are possible within the scope 
of the invention as claimed. 
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1. A video surveillance unit comprising: 

a video camera that includes an adjustable lens and a lens controller, said 
adjustable lens having focus adjusting means and lens-iris adjusting means, and said lens 
controller being operatively connected to the adjustable lens for controlling operation of the 
5 focus adjusting means and the lens-iris adjusting means; and 

a surveillance unit controller operatively connected to said lens controller for 
providing an operation-mode signal thereto, whereby the operational mode of said lens 
controller is selected from a group consisting of an auto-focus mode, an auto-iris mode, a 
combined auto-focus/auto-iris mode, and an external control mode. 
10 2. A video surveillance unit as set forth in Claim 1 wherein the adjustable lens 

comprises a variable f-stop, zoom lens. 

3. A video surveillance unit as set forth in Claim 2 wherein said surveillance unit 
controller comprises means for providing a zoom-adjustment signal with said operation-mode 
signal to said lens controller, whereby the zoom factor of said adjustable lens can be changed 

15 when the operation-mode signal indicates the external control mode of operation for said lens 
controller. 

4. a video surveillance unit as set forth in Claim 2 wherein said surveillance unit 
controller comprises means for providing a focus-adjustment signal with said operation mode 
signal to said lens controller, whereby the focal distance of said adjustable lens can be 

20 changed when the operation-mode signal indicates the external control mode of operation for 
said lens controller. 

5. a video surveillance unit as set forth in Claim 2 wherein said surveillance unit 
controller comprises means for providing a lens-iris-adjustment signal with said operation 
mode signal to said lens controller, whereby the iris opening of said adjustable lens can be 

25 changed when the operation-mode signal indicates the external control mode of operation for 
said lens controller. 

6. A video surveillance unit as set forth in Claim 1 comprising a pan-and-tilt 
apparatus for supporting said video camera and orienting said video camera to one of a 
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plurality of positions each defined by a pan coordinate and a tilt coordinate; and wherein said 
surveillance unit controller comprises: 

pan-and-tilt control means for providing a pan direction 
signal, a tilt direction signal, a pan velocity signal, and a tilt 
5 velocity signal to said pan-and-tilt apparatus, whereby said pan- 

and-tilt apparatus can be driven to orient said video camera to a 
desired position; 

pan-and-tilt position determining means for determining the 
pan and tilt coordinates of the video camera when in the desired 
10 position; and 

first data-storage means for storing the pan and tilt 
coordinates of the video camera. 

7. A video surveillance unit as set forth in Claim 6 wherein said surveillance unit 
controller comprises second data-storage means for receiving and storing a zoom-adjustment 

15 setting from said lens controller in association with the stored pan and tilt coordinates, 
whereby a zoom-adjustment setting is preselected for a set of pan and tilt coordinates. 

8. A video surveillance unit as set forth Claim 6 wherein said surveillance unit 
controller comprises third data-storage means for receiving and storing a focus-adjustment 
setting from said lens controller in association with the stored pan and tilt coordinates, 

20 whereby a focus-adjustment setting is preselected for a set of pan and tilt coordinates. 

9. A video surveillance unit as set forth Claim 6 wherein said surveillance unit 
controller comprises fourth data-storage means for receiving and storing a lens-iris setting 
from said lens controller in association with the stored pan and tilt coordinates, whereby a 
lens-iris setting is preselected for a set of pan and tilt coordinates. 

25 10. A video surveillance unit comprising: 

a video camera that includes an adjustable lens, an imaging array, and an 
imaging controller, said imaging array being positioned to receive an optical image from the 
adjustable lens and being adapted to produce an array signal representing the optical image, 
and said imaging controller being operatively connected to said imaging array for providing a 
30 timing signal for adjusting the shutter speed of said imaging array; and 

a surveillance unit controller operatively connected to the imaging controller for 
providing a shutter-speed signal to said imaging controller, whereby the shutter speed of said 
imaging array can be adjusted. 
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11. A video surveillance unit as set forth in Claim 10 comprising a pan-and-tilt 

apparatus for supporting said video camera and orienting said video camera to one of a 
plurality of positions each defined by a pan coordinate and a tilt coordinate; and wherein said 
surveillance unit controller comprises: 
5 pan-and-tilt control means for providing a pan direction 

signal, a tilt direction signal, a pan velocity signal, and a tilt 
velocity signal to said pan-and-tilt apparatus, whereby said pan- 
and-tilt apparatus can be driven to orient said video camera to a 
desired position; 

10 pan-and-tilt position determining means for determining the 

pan and tilt coordinates of the video camera when in the desired 
position; and 

first data-storage means for storing the pan and tilt 
coordinates of the video camera. 
15 12. A video surveillance unit as set forth Claim 1 1 wherein said surveillance unit 

controller comprises data-storage means for receiving and storing a shutter-speed setting from 
said imaging controller in association with the stored pan and tilt coordinates, whereby a 
shutter-speed setting is preselected for a set of pan and tilt coordinates. 

13. A video surveillance unit as set forth in Claim 11 wherein said video camera 
20 comprises: 

a video processor connected to receive said array signal, for converting said 
array signal into a video signal, and for adjusting the white balance of the video signal 
relative to the array signal; and wherein 

said surveillance unit controller comprises means for providing an operation-mode signal to 
25 said imaging controller whereby the operational mode of the video camera is selected from a 
group consisting of an automatic white-balance mode, a manual white balance mode, and a 
white-balance set mode. 

14. A video surveillance unit as set forth in Claim 13 wherein said surveillance unit 
controller comprises means for providing a white-balance-adjustment signal to said imaging 

30 controller, said video processor comprises a colour processor; and said imaging controller 
comprises means for providing a colour adjustment signal to said colour processor, whereby 
the white balance of the video signal can be adjusted when the operation-mode signal 
indicates the manual white-balance mode of operation for said video camera. 
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15. A video surveillance unit as set forth in Claim 14 wherein said surveillance unit 

controller comprises second data-storage means for receiving and storing a white-balance 
setting from said imaging controller in association with the stored pan and tilt coordinates, 
whereby a white-balance setting is preselected for a set of pan and tilt coordinates. 
5 16. A video surveillance unit comprising: 

a source of an ac power signal; and 

a surveillance unit controller connected to said ac power signal source, said 
surveillance unit controller including a linelock reference generator for generating a linelock 
reference signal in response to the ac power signal; 
10 a video camera that includes an adjustable lens, an imaging array positioned to 

receive an optical image from the adjustable lens and being adapted to produce an array 
signal representing the optical image, and a timing circuit operatively connected to the 
imaging array for providing a timing signal for adjusting the shutter speed of said imaging 
array; and 

15 an imaging controller operatively connected to the surveillance unit controller 

for receiving the linelock reference signal, said imaging controller including delay means for 
providing a synchronization reference signal to said timing circuit, said synchronization 
reference signal having a preselected delay relative to said linelock reference signal. 
17. A video surveillance unit as set forth in Claim 16 wherein said imaging 

20 controller comprises: 

means for storing a delay value corresponding to the preselected delay in the 
synchronization reference signal; 

means for receiving a delay value adjustment signal from said surveillance unit 
controller; and 



25 means for adjusting the delay value stored in said delay value storing means. 

18. A video surveillance system comprising: 

a video surveillance unit as set forth in Claim 17; 
^ a central control unit operatively connected to said video surveillance unit; 

a video switcher/multiplexer connected for receiving a video signal from said 



30 video surveillance unit and for providing the video signal to a display monitor in accordance 
with instructions received said central control unit; 

a synchronization signal stripper connected to receive the video signal provided 
by said video switcher/multiplexer, said synchronization signal stripper being adapted to 
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extract a synchronization component from the video signal and to provide a first 
synchronization signal consisting of the synchronization component of the video signal; 

a signal generator for providing a reference synchronization signal; 

synchronization signal comparison means connected to receive the first 
5 synchronization signal and the reference synchronization signal and provide a phase delay 
adjustment signal to the central control unit, whereby a phase difference between the video 
signal and a reference video signal can be minimized. 

19. a video surveillance system as set forth in Claim 18 wherein said 

synchronization signal comparison means comprises: 
10 a phase comparator responsive to the first synchronization signal and the 

reference synchronization signal for generating a difference signal representing the direction 

and degree of phase difference between the first synchronization signal and the reference 

synchronization signal; 

a filter connected to receive the difference signal from said phase comparator 
15 and to provide an analog signal having an amplitude corresponding to the direction of phase 

difference between the first synchronization signal and the reference synchronization signal; 

and 

a window comparator connected to receive the analog signal from said filter and 
provide a first logic-level signal when the amplitude of the analog signal exceeds an upper 

20 reference amplitude or provide a second logic-level signal when the amplitude of the analog 
signal is below a lower reference amplitude, said first logic-level signal indicating to said 
central control unit that the delay value stored in said imaging controller is to be increased 
and said second logic-level signal indicating to said central control unit the delay value stored 
in said imaging controller is to be decreased, whereby the delay value is adjusted. 

25 20. A video surveillance system as set forth in Claim 19 comprising: 

a first multiplexer connected to receive the first logic-level signal from said 
window comparator in association with an address corresponding to said video surveillance 
unit; 

a second multiplexer connected to receive the second logic-level signal from 
30 said window comparator in association with an address corresponding to said video 
surveillance unit; 

said first and second multiplexers being connected to receive an address from 
• said central control unit and to provide a first or second logic-level signal associated with the 
received address to the central control unit. 
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